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Abstract 

This  study  provides  an  exploratory  view  of  logistical 
support  procedures  for  Defense  Mapping  Agency  (DMA) 
automated  cartographic  or  photogrammetric  systems.  Its 
purpose  was  to  develop  an  Integrated  Logistics  Support  (ILS) 
model  tailored  to  the  requirements  of  DMA  automated 
cartographic  or  photogrammetric  systems. 

The  literature  review  identified  Ostrofsky's  Logistics 
Systems  Management  Matrix  (LSMM)  as  an  acceptable  conceptual 
foundation  for  the  development  of  a  descriptive  logistics 
support  model  for  DMA. 

A  Delphi  survey  provided  the  means  for  constructing  a 
descriptive  model  by  polling  31  DMA  administrative,  research 
and  development,  support  and  maintenance,  and  production 
experts.  ■§: 

Research  findings  supported  the  need  for  DMA  to 
implement  both  a  structured,  ILS  approach  to  automated 
cartographic  or  photogrammetric  systems  planning  and  an 
integrated  organizational  structure  permitting  the  execution 
of  an  ILS  plan.  The  study  also  certified  the  need  to 
determine  and  design  logistics  support  requirements  early  in 
the  system  life  cycle  with  attention  given  to  those  support 
factors  identified  as  unique  to  DMA.  Special  considerations 
should  be  given  to  maintenance  planning  as  the  experts 
determined  this  ILS  element  as  absolutely  critical  to  the 
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effective  logistics  support  of  automated  cartographic  or 
photogrammetric  systems. 


DEVELOPMENT  OF  A  LOGISTICS  SUPPORT  FRAMEWORK 

FOR 

DEFENSE  MAPPING  AGENCY  (DMA)  AUTOMATED  SYSTEMS 

I .  Introduction 

Problem  Background 

Although  the  science  of  cartography  has  been  in 
evidence  for  thousands  of  years,  the  full  extent  of  its 
applications  and  its  value  as  a  source  of  information 
has  never  really  been  fully  exploited.  (Pearce, 

1987:75) 

To  exploit  these  potential  applications  requires  the 
assistance  of  complex  data  processing  systems.  Converting 
from  a  manual  to  an  automated  production  environment 
mandates  parallel  changes  in  organizational  structure  and 
processes  (Daft  and  Steers,  1986:297).  Inherent  to  this 
trend  is  the  need  to  maintain  and  support  automated 
cartographic  or  photogrammetric  systems. 

Within  the  Department  of  Defense  (DOD),  the  agency 
responsible  for  implementing  automated  cartographic  or 
photogrammetric  applications  is  the  Defense  Mapping  Agency 
(DMA).  The  Defense  Mapping  Agency  was  formed  in  1972  from 
various  Army,  Navy,  and  Air  Force  mapping,  charting,  and 
geodetic  organizations.  Today  over  9000  people  provide 
earth  data  (mapping,  charting,  and  geodetic  data)  to  all  the 
military  services  in  support  of  various  weapons  systems 
requirements  (Larson  and  Pelletiere,  1989:81).  Appendix  A 
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lists  the  various  weapons  systems,  by  major  service 

organization,  which  DMA  supports  with  earth  data  products. 

The  requirements  for  DMA's  products  are  generated  by 
the  unified  and  specified  commands  and  the  military 
departments.  Even  though  DMA  is  controlled  by  the 
Assistant  Secretary  of  Defense  for  Command,  Control, 
Communications,  and  Intelligence,  the  Director  of  DMA  ^ 

is  responsible  to  the  Chairman  of  the  Joints  Chiefs  of 
Staff  for  operational  matters.  (Larson  and  Pelletiere, 

1989:12) 

Military  high-technology  weapon  systems  and  flight 

simulators  require  precise  digital  earth  data.  To  respond 

to  these  digital -based  earth  data  systems,  DMA  converted 

from  manual  to  automated  production  processes. 

In  1982,  Congress  endorsed  the  Office  of  the  Secretary 
of  Defense  mandate  that  the  Defense  Mapping  Agency 
should  initiate  a  program  to  develop  an  all-digital 
production  system  capable  of  meeting  the  Armed  Forces' 
critical  demands  for  Mapping,  Charting,  and  Geodetic 
(mcsg)  support.  (Department  of  Defense  ( DOD ) / DMA 
Digital  Production  System  Handbook,  1989:i) 

In  1983,  DMA  initiated  the  ten  year  (1983-1993) 

Exploitation  Modernization  Program  (EMP).  The  purpose  of 

the  $2.6  billion  dollar  EMP  was  to  provide  a.  digital 

production  capability  known  as  the  Digital  Production  System 

(DPS)  which  is  being  developed  in  two  phases,  MARK  85  and 

MARK  90  (See  Appendices  B  and  C).  This  modernized  digital 

production  capability  would  result  in  a  75  percent  decrease 

in  production  pipeline  time  and  a  50  percent  reduction  in 

production  costs  (Larson  and  Pelletiere,  1989:81). 

To  achieve  these  efficiencies,  proper  logistical 

support  for  automated  cartographic  or  photogrammetric 

systems  providing  the  digital  production  capability  is 
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essential.  Operations  and  Maintenance  (O&M)  and  logistical 
support  for  DPS  systems  constitute  a  substantial  amount  of 
the  EMP.  Approximately  25%  of  the  total  allocation  for  the 
EMP,  or  $60C  million  dollars,  was  budgeted  for  logistical 
support  activities  (DOD/DMA  DPS  Maintenance  Summary,  1989). 

To  provide  proper  logistical  support  procedures,  DMA 
must  incorporate  a  life  cycle  management  approach  for  its 
automated  cartographic  or  photogrammetric  systems.  A  life 
cycle  management  approach.  Integrated  Logistics  Support 
(ILS)  is  the  concept  used  to  support  a  product  from  the 
conceptual  phase  to  the  retirement  phase.  ILS  concepts  are 
very  important  in  relation  to  DMA  automated  cartographic  or 
photogrammetric  systems.  Characterized  as  high-technology 
and  fast-obsolescence,  DMA  systems  require  execution  of 
logistical  support  procedures  throughout  the  entire  life 
cycle  of  a  particular  system.  This  research  identified 
principles  of  management,  design  or  system  life  cycle 
processes,  and  ILS  elements  needed  to  implement  and  maintain 
DMA’s  automated  cartographic  or  photogrammetric  systems. 

Technological  changes  in  automated  cartographic  and/or 
photogrammetric  production  processes  require  DMA  to 
continually  re-evaluate  hardware  and  software  requirements 
for  existing  and  new  systems.  The  DPS  provided  DMA  with  the 
latest  automated  cartographic  and  photogrammetric  production 
technology.  Figure  1  illustrates  the  various  DPS  segments 
and  the  complex  interaction  between  the  segments  of  DMA 
automated  production  systems. 
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Figure  1.  DPS  Segments  and  Functional  Flow 

Source:  (DOD/DMA  DPS  Handbook,  1989 :ii) 

The  primary  functions  of  the  DPS  segments  illustrated  in 

Figure  1  are  to  (1)  extract  terrain  elevation  and  feature 

data  from  imagery  on  a  multi-product  basis;  (2)  store  that 

data  in  a  global  data  base  organized  by  data  type  and  levels 

within  data  type;  and  (3)  use  that  data  to  generate  a 

variety  of  graphic  and  digital  earth  data  products. 

The  fully  operational  DPS,  shown  in  Figure  1,  contains 

five  HARK  90  segments  and  two  residual  MARK  85  segments. 

MARK  85,  fully  operational  in  1987,  enhanced  hardcopy 
production  methods,  improved  production  and  data  base 
management,  and  provided  an  initial  softcopy  production 
capability.  Five  years  later,  MARK  90  segments  will 
replace  many  MARK  85  segments  and  provide  full  digital 
softcopy  production  capability  and  improve  production 
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management  in  areas  of  production  programming, 
scheduling,  and  resource  management.  (Department  of 
Defense  (DOD) /DMA  Digital  Production  System  Handbook, 
1989:3) 

Proper  configuration  management  controls  must  be  in  place  to 
ensure  that  hardware/software  modifications  are  coordinated 
with  all  segments  of  the  DPS  system.  Configuration  controls 
for  maintenance  and  support  are  also  required  for  the  entire 
DPS  system.  Proper  logistical  support  procedures  will 
provide  systems  operability  throughout  the  DPS  and  ensure 
DMA's  mission  readiness  in  support  of  user  requirements. 

Problem  Statement 

Although  the  first  phase  (MARK  85)  of  the  DPS  is  fully 
operational,  DMA  lacks  an  integrated  approach  of  formulating 
logistical  support  requirements  for  their  automated  systems. 
MARK  90  and  future  systems  such  as  the  Digital  Geographic 
Information  Tailoring  System  (DIGITS)  require  logistical 
support  planning  procedures  for  on-time  implementation 
(Rivenbark,  1989:8,13). 

Research  Objectives 

This  research  was  initiated  to  determine  how  an 
Integrated  Logistics  Support  (ILS)  or  other  logistical 
support  planning  model  can  provide  a  framework  for 
determining  logistical  support  requirements  for  DMA 
automated  cartographic  or  photogrammetric  systems.  Specific 
research  objectives  addressed  were: 


1.  Develop  an  ILS  or  similar  logistical  support  model 
to  address  the  logistical  support  requirements  for  automated 
cartographic  or  photogrammetric  systems. 

2.  Validate  the  model  by  determining  its  relevance  to 
DMA  automated  cartographic  or  photogrammetric  systems. 

Research  Questions 

To  fulfill  the  research  objectives,  the  following 
investigative  questions  were  addressed: 

1.  Can  DMA  experts  (administrative,  research  and 
development,  production,  operations  support  and  maintenance) 
reach  a  level  of  consensus  on  the  characteristics  and 
attributes  of  a  logistics  support  model  applicable  to 
automated  cartographic  or  photogrammetric  systems? 

2.  What  factors  within  the  ILS  elements  are  unique  and 
applicable  to  automated  cartographic  or  photogrammetric 
systems? 

3.  At  what  stages  of  the  automated  system's  life  cycle 
do  the  ILS  elements  require  determination  and  design? 


Definitions 


1.  Cartography:  The  art  and  science  of  making  maps 
and  charts,  globes,  and  relief  models.  (Raisz, 
1962:293) 

2.  Earth  Data:  Quantified  and  codified  information 
about  the  earth  and  the  features  on  its  surface. 

The  more  common  terminology  for  earth  data  is 
mapping,  charting,  and  geodetic  data  (MC&G) .  Maps 
are  graphic  representations  of  the  land  areas  of 
the  earth.  Charts  are  used  for  both  nautical  and 
aeronautical  navigation.  The  two  types  of  geodetic 
data  of  greatest  significance  to  the  military  are 
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the  precise  locations  of  points  on  the  earth  and 
gravity  measurements.  (Larson  and  Pelletiere, 
1989:13) 

3.  MARK  85:  MARK  85  was  the  first  phase  of  the 
Digital  Production  System  (DPS)  and  consisted  of 
six  segments.  MARK  85  was  a  transitional  step 
which  upgraded  DMA's  production  system  and  provided 
initial  digital  exploitation  capability.  MARK  *  8  5 
has  been  delivered  and  is  in  use  at  DMA  production 
centers.  See  Appendix  B  for  a  more  detailed  review 
of  MARK  85.  (DOD/DMA  DPS  Handbook,  1989: i) 

4.  MARK  90:  Mark  90  is  the  second  phase  of  the 
Digital  Production  System  (DPS)  and  consists  of 
five  MARK  90  segments  and  the  residual  of  two  MARK 
85  segments.  MARK  90  will  provide  an  all-digital 
softcopy  production  system  which  will  improve 
production  efficiencies  and  accommodate  mandatory 
conversion  to  exploitation  of  improved  source 
materials.  In  March  1991,  the  MARK  90  system  will 
reach  its  Initial  Operating  Capability  (IOC).  Full 
Operating  Capability  (FOC)  will  occur  in  March  1993 
for  all  31  products  in  the  DPS  baseline.  See 
Appendix  C  for  a  more  extensive  treatment  of  M^RK 
90.  (DOD/DMA  DPS  Handbook,  1989: ii) 

5.  Logistics:  The  art  and  science  of  management, 
engineering,  and  technical  activities  concerned 
with  requirements,  design,  and  supplying  and 
maintaining  resources  to  support  objectives,  plans, 
and  operations.  (How  SOLE  Defines  Logistics, 
1974:18-19) 

6.  Logistician:  An  individual  whose  profession  or 
specialty  is  performing  one  of  more  of  the  prime 
management  functions  (planning,  organizing, 
coordinating,  directing,  and  controlling)  in  a 
logistics  discipline  or  functional  area  or  who  is 
responsible  for  ensuring  logistics  processes  are 
completed  in  support  of  an  organization's 
activities.  (Gregor,  1988:8) 

7.  Integrated  Logistics  Support:  A  disciplined, 
unified,  and  iterative  approach  to  the  management 
and  technical  activities  necessary  to: 

(a)  integrate  support  considerations  into  system 
and  equipment  design;  (b)  develop  support 
requirements  that  are  related  consistently  to 
readiness  objectives,  to  design,  and  to  each  other; 
(c)  acquire  the  required  support;  (d)  provide  the 
required  support  during  the  operational  phase  at 
minimum  cost.  (DOD  Directive  5000.39,  1983:2-2) 
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Scope  and  Limitations 


Using  the  Delphi  approach,  35  DMA  administrative, 
research  and  development,  production,  operations  support  and 
maintenance  experts  were  chosen  to  represent  the  population 
of  users  associated  with  automated  cartographic  or 
photogrammetric  systems.  The  survey  population  was  limited 
to  personnel  of  the  Defense  Mapping  Agency  because  of  their 
responsibility  in  providing  graphic  and  digital  cartographic 
and  photogrammetric  products  for  all  the  military  services 
(Larson  and  Pelletiere,  1989:12).  The  descriptive  model 
built  was  validated  and  modified  based  on  Delphi  results 
gathered  from  DMA  expert  opinions. 

The  principles  of  management,  depicted  in  the  Logistics 
Systems  Management  Matrix  ( LSMM)  portrayed  in  Chapter  II, 
received  limited  treatment  because  the  body  of  management 
research  was  far  too  extensive  to  review  in  the  course  of 
this  research  effort.  The  major  focus  of  the  research 
effort  concentrated  on  the  application  of  design  or  life 
cycle  phases  and  Integrated  Logistics  Support  (ILS)  elements 
of  the  LSMM  to  automated  cartographic  or  photogrammetric 
systems . 

Conclusion 

The  Defense  Mapping  Agency  plays  an  important  role  in 
the  development  and  deployment  of  many  major  service  weapons 
systems  by  providing  up-to-date  and  accurate  earth  data.  To 
meet  the  requirements  of  these  modern  weapons  systems,  DMA 
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made  dynamic  changes  in  their  production  processes.  Manual 
techniques  were  replaced  by  automated  techniques  for  all 
possible  operations.  A  $2.6  billion  dollar  Exploitation 
Modernization  Program  (EMP)  within  DMA  was  initiated  to  meet 
future  military  requirements.  With  logistical  support 
constituting  25%  or  $600  million  dollars  of  the  total 
allocation  for  the  Exploitation  Modernization  Program,  the 
importance  of  determining  logistical  support  procedures 
early  in  the  life  cycle  of  DMA  systems  is  mandatory. 

Research  objectives  and  research  questions  were  developed  to 
examine  the  special  logistical  support  needs  of  automated 
cartographic  or  photogrammetric  systems. 

In  Chapter  II,  a  review  of  the  literature  presents 
Ostrofsky's  Logistics  Systems  Management  Matrix  ( LSMM)  as  a 
model  for  determining  logistical  support  requirements  for 
DMA  automated  cartographic  or  photogrammetric  systems. 
Chapter  III  defines  the  Delphi  methodology  which  provides 
DMA  "experts"  the  opportunity  to  furnish  data  required  to 
develop  a  logistics  support  model  applicable  to  automated 
cartographic  or  photogrammetric  systems.  Chapter  IV 
presents  the  results  from  two  iterations  of  the  Delphi 
survey.  Chapter  V  summarizes  the  findings  to  the  research 
questions  and  develops  a  descriptive  model  to  address  the 
logistical  support  requirements  for  automated  cartographic 
or  photogrammetric  systems.  Chapter  VI  presents 
contributions  of  this  research  and  recommendations  for 
future  research. 
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II .  Literature  Review 

The  identification  and  application  of  unique  logistics 
support  factors  throughout  the  entire  life  cycle  of  DMA 
automated  cartographic  or  photogrammetric  systems  is 
mandatory  to  ensure  the  mission  readiness  of  all  DOD 
components.  Based  on  Ostrof sky's  Logistics  Systems 
Management  Matrix  ( LSMM) ,  this  literature  review  examined 
the  model's  three  dimensions  and  their  interrelationships. 
The  LSMM,  shown  in  Figure  2,  was  used  as  the  foundation  for 
the  development  of  a  logistics  model  applicable  to  automated 
cartographic  or  photogrammetric  systems.  The  concepts 
derived  from  this  literature  review  were  used  to  form  the 
Delphi  survey  questions  in  Chapter  III. 

As  shown  in  Figure  2,  the  LSMM  is  a  three-dimensional 
model  with  the  principles  of  management  on  the  "X"  axis, 
design  or  system  life  cycle  processes  on  the  "Y"  axis,  and 
the  ILS  elements  on  the  "Z"  axis.  The  LSMM  forms  the  basis 
for  proper  implementation  of  a  logistics  support  plan  for 
any  given  system  (Ostrofsky,  1986:8). 

Several  modifications  to  Ostrofsky's  model  were  made  to 
implement  logistics  support  procedures  for  automated 
cartographic  or  photogrammetric  systems.  Staffing  was 
eliminated  from  Ostrofsky's  principles  of  management  axis  in 
the  matrix.  Most  classical  management  texts  reference 
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Figure  2.  The  Logistics  Systems  Management  Matrix 

(Ostrofsky,  1986:8) 
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planning,  organizing,  controlling,  and  directing  as  the 

principles  of  management.  Daft  and  Steers  stated 

...  management  is  defined  as  the  process  of  planning, 
organizing,  directing,  and  controlling  the  activities 
of  employees  in  combination  with  other  organizational 
resources  to  accomplish  stated  organizational 
objectives.  (Daft  and  Steers,  1986:15) 

Substitutions  for  Ostrof sky's  design  or  system  life  cycle 

factors  and  the  ILS  elements  were  made  to  conform  with  DOD 

specifications  found  in  DOD  5000.39.  These  changes  were 

acceptable  as  long  as  the  entire  system  life  cycle  was  shown 

and  the  ILS  elements  were  compatible  with  the  particular 

system  application  (Ostrofsky,  1986:7). 

The  literature  review  identified  the  LSMM  as  a  suitable 

framework  for  determining  logistical  support  requirements 

for  automated  cartographic  or  photogrammetric  systems.  A 

limited  treatment  of  the  principles  of  management  depicted 

on  the  "X"  axis  of  the  LSMM  was  conducted  because  the  body 

of  management  research  was  far  too  extensive  to  review  in 

the  course  of  this  research  effort.  Rather,  the  thrust  of 

the  literature  review  focused  on  life  cycle  logistics 

support  applications  for  automated  cartographic  or 

photogrammetric  systems.  The  literature  review  included 

(1)  the  identification  of  the  six  basic  design  or  life  cycle 

phases;  (2)  the  identification  of  the  ten  ILS  elements;  and 

(3)  the  relationship  of  the  ILS  elements  as  they  advance 

through  the  design  or  life  cycle  phases  of  a  system. 


Principles  of  Management 


Planning,  organizing,  controlling,  and  directing  are 
the  four  principles  of  management  studied  in  most  classical 
management  texts.  "Fundamental  to  the  management  of 
logistics  as  with  any  system,  is  the  comprehension  of  the 
principles  of  management  and  the  nature  of  the  decisions 
which  require  resolution"  (Ostrofsky,  1986:7). 

Planning  is  the  first  phase  of  the  management  process. 
Planning  involves  what  is  to  be  done,  when  to  do  it,  who 
will  do  it,  and  the  procedures  for  accomplishing  it 
(Steiner,  1962:28).  Over  sixty  years  ago,  Henri  Fayol 
pointed  out  that  managing  means  looking  ahead  and  that 
planning  is  essential  in  the  business  environment  (Fayol, 
1949:43).  These  ideas  are  still  applicable  in  present  day 
organizations.  Systems  planning  still  must  be  an  integral 
part  of  the  strategies,  policies,  and  business  needs  of  any 
organization's  logistics  operation  (Ernst  &  Whinney, 
1987:70). 

The  field  of  cartography/photogrammetry  is  changing  at 
an  alarming  rate  requiring  DMA  to  plan  effectively  both  for 
the  short  and  long-term.  Manual  production  processes  have 
given  way  to  automated  techniques  and  applications  utilizing 
state-of-the-art  computer  hardware  and  software  (Larson  and 
Pelletiere,  1989:81).  The  attributes  (i.e.  military 
requirements,  high  performance  graphics  hardware,  optics, 
electronics)  of  these  systems  continually  change  due  to 
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advancements  in  computer  technologies  (Larson  and 
Pel  1 etiere ,  1989:81). 


Organizing  is  the  management  principle  that  achieves  * 

coordination  through  a  series  of  tasks  and  relationships 

(Matteson  and  Ivancevich,  1989:83).  Organizing  can  be 

further  broken  down  in  particular  organizational  types 

within  a  particular  environment.  In  high  tech  environments, 

there  are  two  primary  organizational  types.  These  are 

. . .  functional  organizations  which  are  responsible  for 
administration,  policy  and  procedures,  personnel 
guidance,  maintaining  technical  excellence,  and 
providing  skilled  personnel...  and  program 
organizations  which  are  responsible  for  directing 
program  activity,  developing  and  maintaining  program 
plans,  developing  cost  and  schedule  parameters, 
evaluating  progress  and  status,  forecasting,  and 
reporting.  (Guthrie,  1984:D.1.4) 

Knowledge  of  the  primary  organizational  components  is 
imperative  for  DMA  logistics  managers  designing  support 
procedures  for  automated  cartographic  or  photogrammetric 
systems.  Automated  cartographic  or  photogrammetric  systems 
consist  of  "highly  complex  computer  oriented  systems  which 
include  system  design,  software  development,  hardware 
fabrication  and  assembly  and  subsequent  operations  and 
maintenance  on  the  final  product"  (Guthrie,  1984:D.1.1). 

These  systems  require  interaction  between  organizational 
components  within  DMA  to  accomplish  multiple  interactive  * 

processes.  DMA's  attempt  to  facilitate  this  interaction  is 
illustrated  by  its  organizational  structure  provided  in  , 

Appendix  D. 
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Controlling  is  the  management  principle  that  is  least 


understood  and  is  required  to  "insure  that  plans  and 
organizations  are  executed  in  a  manner  consistent  with 
achieving  the  desired  results"  (Matteson  and  Ivancevich, 
1989:84).  The  means  for  establishing  criteria  and  measuring 
these  criteria  are  mandatory  when  developing  logistics 
support  procedures  for  automated  cartographic  or 
photogrammetric  systems.  Logistical  support  data  measured 
throughoi.';  the  systems'  life  cycle  must  be  maintained  by 
some  configuration  control  process.  The  process  allows  the 
logistics  manager  to  interact  with  other  components 
(engineering,  production)  to  control  and  maintain  the 
integrity  of  the  system  throughout  the  design  or  life  cycle. 

Directing  is  the  last  of  the  management  principles 
addressed  in  this  literature  review.  Synonymous  with 
directing  is  the  work  by  Fiedler  regarding  contingency 
theory.  Contingency  theory,  in  leadership  terms,  "specifies 
that  a  group's  performance  depends  upon  both  leadership 
style  and  the  nature  of  the  leadership  situation"  (Daft  and 
Steers,  1986:411).  Contingency  theory  may  be  applied  to 
logistics  management  in  a  similar  context.  Germain's 
research  states 

. . .  that  management  does  alter  logistical  strategy  and 
structure  in  response  to  product  customization. 

Managers  appear  to  (1)  consolidate  fewer  logistical 
activities  in  a  single  department  (2)  rely  less  on  a 
formalized  logistics  system  and  (3)  use  shorter  time 
periods  between  updating  their  strategic  logistics 
plan.  In  general,  these  findings  are  in  agreement  with 
contingency  theory,  or  the  premise  that  organizations 
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are  open  systems  affected  by  their  environment. 

(Germain,  1989:21) 

Recent  organizational  changes  within  DMA  support 
Germain's  research  concerning  directing  or  contingency 
theory.  DMA  altered  their  logistical  strategy  and  structure 
in  response  to  changing  trends  in  automated  production 
processes.  The  creation  of  a  new  Directorate  for 
Acquisition,  Installation,  and  Logistics  provides  a 
direction  for  identifying  and  implementing  logistical 
support  requirements  for  automated  cartographic  or 
photogrammetric  systems. 

The  application  of  these  four  principles  of  management 
forms  the  decision  making  process  that  will  take  place 
throughout  the  design  or  life  cycle  of  automated 
cartographic  or  photogrammetric  systems  at  DMA  (Ostrofsky, 
1986:7) . 

Design  or  System  Life  Cycle 

The  second  dimension  of  the  LSMM  "reinforces  the 
establishment  of  formal  areas  to  be  considered  in  the  system 
life  cycle"  (Ostrofsky,  1986:8).  The  specific  phases  of  the 
system  life  cycle  used  in  this  research  differ  slightly  from 
Ostrofsky's  LSMM  although  the  complete  system  life  cycle  is 
maintained.  The  design  or  system  life  cycle  phases  selected 
parallel  DOD  life  cycle  phases  while  providing  a  less  formal 
approach  to  be  used  by  DMA  experts  in  the  Delphi  survey. 

The  six  basic  design  or  system  life  cycle  phases  to  be  used 
are  illustrated  in  Table  1. 
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Table  1 


Design  or  System  Life  Cycle  Phases 


1.  Conceptual  Phase 

2.  Advanced  Development  Phase 

3.  Detailed  Design  and  Development  Phase 

4.  Production  Phase 

5.  Operational  Use  Phase 

6.  Retirement  Phase 


(Finkelstein  and  Guertin,  1988:199) 


Finkelstein  summarize-  the  six  basic  design  or  system 

life  cycle  phases  as  follows 

.  .  during  the  conceptual  phase  a  product  or  system  is 
defined,  acquisition  requirements  are  determined, 
production  and  operational  support  requirements  are 
identified  and  logistics  support  planning  is 
established.  During  the  advanced  development  phase  the 
product  objectives,  performance  limits,  areas  of  risk, 
alterations  and  acquisition  methodologies  are 
determined.  Logistics  support  is  performed  and  the 
logistics  support  plan  is  initiated.  During  the 
detailed  design  and  development  phase  the  product  is 
design  and  tested  and  the  effects  on  reliability, 
maintainability,  and  logistics  support  are  determined. 
The  production  phase  consists  of  producing  the  product 
and  its  support  and  test  equipment,  spare  parts, 
training,  software,  and  packaging,  handling  & 
transportation.  The  operational  use  phase  includes  all 
activities  associated  with  operational  use  and 
logistics  support.  Retirement  consists  of  all  actions 
necessary  to  phase  out  and  dispose  of  a  system. 
(Finkelstein  and  Guertin,  1988:199) 

The  function  of  ILS,  as  defined  in  Chapter  I,  is  to 
maximize  system  readiness  using  detailed  support  elements 
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throughout  the  system's  life  cycle.  Logistics  Support 

Analysis  (LSA)  is  an  important  process  to  ensure  the 

effective  implementation  of  Integrated  Logistics  Support 

(ILS)  for  any  system.  LSA  is  defined  as: 

The  selective  application  of  scientific  and  engineering 
efforts  undertaken  during  the  acquisition  process,  as 
part  of  the  system  engineering  and. design  process,  to 
assist  in  complying  with  supportability  and  other  ILS 
objectives.  (Kankey,  1986:4) 

LSA  allows  the  logistics  manager  to  examine  and  define 

logistics  support  requirements,  estimate  logistics  support 

costs,  and  evaluate  logistics  alternatives  (Comptroller 

General,  1980:6). 

ILS/LSA  define  many  of  the  logistics  support  processes 
that  must  take  place  early  in  the  life  cycle  of  a  system. 

To  ensure  total  support  throughout  the  entire  life  cycle  of 
a  system,  an  aggregate  of  the  ILS  elements  is  necessary 
(Finkelstein  and  Guertin,  1988:6). 

Retirement  is  an  integral  component  and  function  of  the 
system  life  cycle  and  must  be  considered  early  in  the 
conceptual  and  advanced  development  phases  (Finkelstein  and 
Guertin,  1988:204).  Fast  obsolescence  systems,  such  as 
automated  cartographic  or  photogrammetric  systems,  require 
retirement  planning  early  in  the  life  cycle.  ILS 
requirements  determination  for  existing  systems  and  future 
systems  are  driven  by  the  ability  to  predict  retirement 
periods  for  given  systems. 
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Integrated  Logistics  Support  ( ILS)  Elements 

The  ten  ILS  elements  selected  for  this  research  come 
directly  from  DODD  5000.39  and  are  illustrated  in  Table  2. 

Table  2 

Integrated  Logistics  Support  (ILS)  Elements 


1.  Maintenance  Planning 

2.  Manpower  and  Personnel 

3.  Supply  Support 

4.  Support  Equipment 

5.  Technical  Data 

6.  Training 

7.  Computer  Resources  Support 

8.  Facilities 

9.  Packaging,  Handling,  Storage  &  Transportation 

10.  Design  Interface 

(DODD  5000.39) 


Although  automated  cartographic  or  photogrammetric  systems 
are  smaller  acquisition  programs  than  perhaps  a  large  weapon 
system  employed  by  the  Air  Force,  the  evaluation  of  their 
ILS  elements  remains  the  same  (Materna  and  Andrews, 
1988:5-4).  Many  times  the  ILS  elements  are  managed 
separately,  but  the  integration  of  the  remaining  elements 
must  be  considered  when  applying  the  principles  of 
management  to  logistics  support  tasks  for  automated 
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cartographic  or  photogrammetric  systems.  Another  important 
concept  in  determining  logistics  support  requirements  for 
automated  cartographic  or  photogrammetric  systems  is  the 
relationship  between  the  ILS  elements  and  the  system  life 
cycle. 

ILS  El ements/ System  Life  Cycle  Relationship 

Integration  of  ILS  elements  and  system  life  cycle  is 
required  for  successful  implementation  of  any  system.  To 
conduct  a  systematic  literature  review,  each  ILS  element 
will  be  analyzed  as  it  advances  through  the  phases  of  the 
system  life  cycle.  A  figure  will  accompany  each  ILS  element 
illustrating  events  which  take  place  during  phases  of  the 
system  life  cycle. 

Maintenance  Planning.  This  ILS  element  is  the  first  of 
ten  elements  a  logistics  manager  must  evaluate.  Maintenance 
planning  is  "the  process  to  develop  the  maintenance  concepts 
and  maintenance  requirements  of  the  system"  (ILS  Guide, 
1986:7-5).  Figure  3  illustrates  the  maintenance  planning 
element  as  it  advances  through  the  system  life  cycle  phases. 

The  maintenance  concept  is  the  first  step  in  the 
maintenance  planning  process  and  is  developed  during  the 
conceptual  phase  of  the  system  life  cycle.  The  maintenance 
concept  is  a  statement  of  guidelines  used  in  developing  the 
maintenance  plan  for  a  system  or  item  of  equipment  (Jones, 
1987:65) . 
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Figure  3.  Maintenance  Planning/Life  Cycle  Relationship 


Jones  states  that  the  guidelines  addressed  by  the 

maintenance  concept  include 

...  (1)  a  strategy  for  allocating  maintenance  tasks  to 
the  different  levels  of  rr='i nt“r  »rce;  (?)  the  repair 
policy  regarding  similar  types  of  items  contained  in 
the  equipment;  (3)  the  criteria  for  scheduling 
maintenance  tasks;  and  (4)  the  availability  of 
resources  to  support  maintenance.  (Jones,  1987:65) 

This  concept  must  agree  with  the  operational  requirements 

and  financial  constraints  of  the  system  (Finkelstein  and 

Guertin,  1988:102).  The  concept  requires  a  careful 

examination  of  the  complete  structure  of  the  system.  The 

maintenance  planning  concept  is  the  "lead  analytical 

activity  and  provides  input  to  the  development  of  all  of  the 

remaining  ILS  elements"  (ILS  Guide,  1986:7-5). 

As  the  system  advances  through  the  life  cycle, 

maintenance  planning  is  updated.  The  maintenance  concept  is 
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compared  with  the  projected  resources  and  constraints  that 

may  be  present  when  the  system  or  equipment  is  fielded 

(Jones,  1987:65).  Specific  maintenance  tasks  are  identified 

in  conjunction  with  operational  requirements  during  the 

advanced  development  phase  and  the  detailed  design  and  # 

development  phase. 

At  the  end  of  the  detailed  design  and  development  phase 
or  early  in  the  production  phase  a  detailed  maintenance  plan 
is  created  (Finkelstein  and  Guertin,  1988:110). 

"Maintenance  planning  identifies  the  level  of  maintenance  at 
which  each  task  is  performed,  the  tools  and  equipment 
required,  and  the  times  and  frequencies  (ILS  Guide,  1986: 

7-5).  The  maintenance  planning  document  created  at  this 
stage  is  updated  throughout  the  life  cycle  as  needed. 

Manpower  and  Personnel .  This  ILS  element  requires 
identifying  personnel  needed  to  operate  and  support  the 
system  throughout  the  life  cycle.  Figure  4  illustrates  the 
manpower  and  personnel  element  as  it  advances  through  the 
system  life  cycle  phases. 

Identification  of  the  skills  necessary  and  the  manpower 
constraints  are  required  prior  to  the  initiation  of  the 
program  (ILS  Guide,  1986:7-5).  During  the  conceptual  phase 
the  first  estimate  of  needed  manpower  criteria  is  performed.  * 

These  initial  estimates  can  be  based  on  historical  data  from 
prior  systems  that  are  similar  to  the  system  being  developed 
(ILS  Guide,  1986:3-4).  There  are  also  models  which  can  be 
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Figure  4.  Manpower  and  Personnel (M/P)/Life  Cycle  Relationship 


procured  for  estimating  manpower  and  personnel  requirements. 
"HARDMAN,  is  an  analytical  tool  which  predicts  quantitative 
manpower  and  personnel  requirements  in  different  skills" 

(ILS  Guide,  1986:7-5) . 

To  ensure  the  required  support  is  maintained 
throughout  the  system  life  cycle,  the  quantity,  type,  and 
appropriate  skill  levels  of  maintenance  personnel  must  be 
attained  (Finkelstein  and  Guertin,  1988:114).  Late  in  the 
detailed  design  and  development  phase  this  information  is 
more  attainable  and  precise  manpower  and  personnel 
requirements  can  be  formulated.  These  fine-tuned 
requirements  are  the  foundation  to  ensure  proper 
implementation  after  the  production  phase  and  into  the 
operational  use  phase.  Adjustments  to  manpower  and 


personnel  can  be  made  according  to  operational  data  and 
production  requirements  throughout  the  remaining  life  cycle. 

Supply  Support .  This  element  encompasses  all  actions 
required  to  identify  and  obtain  the  spares  and  repair  parts 
needed  to  implement  a  given  system  (ILS  Guide,  1986:7-8). 
Figure  5  illustrates  the  supply  support  element  as  it 
advances  through  the  system  life  cycle  phases. 

Supply  support  is  initiated  during  the  advanced 
development  phase  of  the  system  life  cycle.  Due  to  the 
rapid  technological  advancements  in  the  computer 
environment,  parts  obsolescence  is  an  important  concept 
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Figure  5.  Supply  Support(SS)/Life  Cycle  Relationship 
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associated  with  supply  support.  During  the  advanced 
development  phase  planning  must  take  place  to  ensure  parts 
obsolescence  will  not  be  a  major  problem  later  in  the  system 
life  cycle.  There  is  no  real  solution  to  the  problem,  but 
k  exchanging  information  between  logistics  support  personnel 

and  designers  early  in  the  life  cycle  will  assure  that  the 
problem  is  not  unsuspected  (Stolinski,  1987:88). 

Initial  provisioning  begins  early  in  the  detailed 
design  and  development  phase.  Provisioning  in  the  future 
will  change  due  to  advancements  in  automated  systems 
development  (Klein,  1989:37-38).  The  requirement  to  provide 
spare  parts  for  rapidly  changing  systems  will  test  logistics 
managers  in  the  future.  Before  transition  into  the 
production  phase  final  provisioning  and  associated  supply 
support  documentation  should  be  complete.  Revisions  to 
supply  support  continue  throughout  the  entire  life  cycle  in 
the  form  of  hardware  reconfigurations  and  operating  data 
experience . 

Support  Equipment .  This  element  takes  into  account  the 
needed  equipment  to  support  the  operations  and  maintenance 
of  the  system  (ILS  Guide,  1986:7-9).  Figure  6  illustrates 
the  support  equipment  element  as  it  advances  through  the 
system  life  cycle  phases. 

Support  equipment  is  an  integral  part  of  the  complete 
*  system  and  must  be  planned  and  developed  during  the 

conceptual  phase.  The  decision  whether  to  purchase  new 
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Figure  6.  Support  Equipment (SE)/Life  Cycle  Relationship 


support  equipment  or  use  existing  equipment  must  be  made 
early  in  the  system  life  cycle  (ILS  Guide,  1986:7-9). 

Special  support  equipment  should  be  identified  midway 
through  the  detailed  design  and  development  phase.  At  this 
time  "detailed  task  analyses  and  documentation  are  performed 
to  identify  the  specific  equipment  requirements  for  every 
operating  and  maintenance  task"  (ILS  Guide,  1986:7-9). 

Technical  Data .  This  element  provides  all  the 
necessary  technical  information  concerning  the  system  being 
developed.  Operations  and  maintenance  manuals, 
specifications,  drawings,  and  computer  support  material  are 
included  in  this  element.  Figure  7  illustrates  the 
technical  data  element  as  it  advances  through  the  system 
lift  cycle  phases. 
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Figure  7.  Technical  Data(TD)/Life  Cycle  Relationship 

Requisites  for  technical  data  should  be  identified  in 
the  conceptual  phase.  Both  technical  data,  relating  to 
operations  and  maintenance,  and  management  data,  to  be  used 
for  audit  procedures,  must  be  developed  during  the  advanced 
development  phase  (Finkelstein  and  Guertin,  1988:136). 
Preliminary  manuals  covering  these  types  of  data  must  be 
accessible  late  in  the  advanced  development  phase  to  support 
testing  and  training  functions  (ILS  Guide,  1986:7-10).  A 
formal  review  process  of  the  technical  data  is  necessary  to 
ensure  compliance  with  requirements  set  forth  early  in  the 
conceptual  phase.  Updates  to  the  technical  data  associated 
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with  technological  advancements  to  the  system  are  made 
throughout  the  life  cycle. 

Training .  The  training  element  involves  the  methods, 
techniques,  and  resources  used  to  prepare  personnel  to 
operate  and  maintain  a  given  system.  Figure  8  illustrates 
the  training  element  as  it  advances  through  the  system  life 
cycle  phases.  "DODD  5000.39  specifies  that  detailed 
descriptions  of  current  and  projected  skill  and  training 
resources  be  developed  during  the  conceptual  phase"  (ILS 
Guide,  1986:7-10). 
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Figure  8.  Training/Life  Cycle  Relationship 
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Requirements  for  training  and  equipment  are  performed 
during  advanced  development  and  detailed  design  and 
development  phases  (ILS  Guide,  1986:7-10).  The  logistics 
manager  must  work  with  training  personnel  to  ensure  support 
requirements  are  outlined  in  the  training  material.  The 
training  documents  shall  be  reviewed  and  tested  during  the 
later  part  of  the  detailed  design  and  development  phase  or 
early  in  the  production  phase.  The  training  documents 
should  be  free  of  any  discrepancies  when  applied  in  the 
operational  use  phase.  A  cadre  of  operations  and 
maintenance  support  personnel  should  be  designated  as 
follow-on  trainers  for  additional  personnel  employed  on  the 
system  after  initial  training  is  conducted  (DOD/DMA  DPS 
Handbook,  1989:65). 

Computer  Resources  Support .  The  computer  resources 
support  element  can  be  described  as  containing  all  hardware, 
software,  documentation,  contract  support,  personnel,  and 
supplies  required  to  operate  and  maintain  a  given  automated 
system.  With  systems  becoming  increasingly  more  complex, 
this  ILS  element  requires  more  attention.  Figure  9 
illustrates  the  computer  resources  support  element  as  it 
advances  through  the  system  life  cycle  phases. 

Both  software  and  hardware  development  should  begin 
early  in  the  conceptual  phase  of  the  system  life  cycle. 
"Logistics  engineers  must  begin  to  express  their  concerns  as 
early  as  the  idea  stage  of  a  program"  (Klein,  1989:37). 
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Figure  9.  Computer  Resources  Support (CRS)/Life  Cycle  Relationship 


During  the  conceptual  phase  and  the  advanced  development 
phase  system  concepts  are  developed  and  system/software  and 
system/hardware  requirements  are  defined  and  examined  (Tile, 
1987:238).  At  the  beginning  of  the  detailed  design  and 
development  phase  hardware  and  software  design  and 
development  are  divided  and  accomplished  as  separate  tasks. 
Hardware  is  designed  to  meet  system  specifications  outlined 
in  the  conceptual  phase  and  the  logistics  manager  must 
ensure  that  the  hardware  design  is  congruent  with  other  ILS 
elements.  In  the  past,  less  attention  was  given  to  software 
supportability  issues  for  automated  systems  than  was  given 
to  hardware  supportability  issues  (Tile,  1987:233). 
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The  latest  research  indicates  that  software  developers 
are  spending  more  time  fixing  software  rather  than 
developing  software  (Haley,  1987:23).  "Corporate 
programmers...  now  spend  80%  of  their  time  just  repairing 
software  and  updating  it  to  keep  it  running"  (Klein, 
1989:38).  These  reasons  amplify  why  software  support  and 
the  logistics  managers  must  be  part  of  the  total  design  of 
the  system  during  the  detailed  design  and  development  phase. 
The  same  logistics  considerations  (modularity, 
transportability,  and  compatibility)  that  are  designed  for 
hardware  must  be  designed  for  software  (Finkelstein, 
1988:204-205).  Standard  operating  procedures  and 
configuration  control  of  software  and  hardware  subsystems 
must  be  developed  and  implemented  during  the  detailed  design 
and  development  phase. 

During  the  production  phase  and  into  the  operational 
use  phase  adherence  to  the  system  configuration  baseline 
provides  the  basis  for  future  modifications  (Finkelstein  and 
Guertin,  1988:140).  If  changes  are  made  to  the  baseline 
configuration,  logistics  managers  must  ensure  proper 
documentation  and  software  are  updated  to  maintain  effective 
system  supportability . 

Faci 1 ities .  The  facilities  element  has  varying 
applications  for  governmental  and  private  organizations.  In 
the  government,  the  facilities  element  captures  both  the 
real  property  assets  required  to  support  the  system  and  the 
studies  which  define  types  of  facilities  or  facility 
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improvements,  locations,  spare  needs,  etc.  (ILS  Guide, 

1986:7-10).  In  private  organizations,  facilities  consist  of 
distribution  systems/centers  and  warehousing  operations. 

Figure  10  illustrates  the  facilities  element  as  it  advances 
through  the  system  life  cycle  phases.  » 
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Figure  10.  Facilities/Life  Cycle  Relationship 


Determination  of  facilities'  requirements  in  the 
private  sector  takes  place  early  in  the  system  life  cycle 
because  it  is  important  to  the  distribution  of  a  product  to 
the  customer  (Finkelstein  and  Guertin,  1988:148).  In  the 
government,  initial  facility  requirements  are  analyzed 
during  the  conceptual  phase.  This  process  must  take  place 
early  in  the  system  life  cycle  due  to  the  constraints  of  the 
Six  Year  Program  Objective  Memorandum  (POM). 
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During  the  advanced  development  phase  the  facilities 
planning  function  takes  place.  Input  factors  of  the 
facilities  planning  function  are  existing  facility  data, 
projected  space  availability,  funding  constraints,  and 
operations  and  maintenance  concepts  (ILS  Guide,  1986:7-11). 
Updates  to  the  facilities  element,  throughout  the  system 
life  cycle,  take  place  when  system  configuration  baseline 
changes  warrant. 

Packaging .  Handling .  Storage  and  Transportation.  The 
packaging,  handling,  storage  and  transportation  (PHS&T) 
element  "includes  the  characteristics,  action  and 
requirements  necessary  to  insure  the  capability  to 
transport,  preserve,  package,  and  handle  equipment  and 
support  items"  (ILS  Guide,  1986:7-11).  Figure  11 
illustrates  the  PHS&T  element  as  it  advances  through  the 
system  life  cycle  phases. 

During  the  conceptual  phase  transportation  requirements 
are  analyzed  against  system  objectives.  Packaging  and 
handling  constraints  for  the  system  are  also  identified 
during  the  conceptual  phase. 

During  the  advanced  development  phase  specific 
transportation  attributes  are  identified  to  support  system 
delivery  after  the  production  phase.  Modes  of 
transportation  and  carrier  selection  are  criteria  requiring 
special  attention  because  of  governmental  deregulation  of 
the  transportation  industry.  During  the  advanced 
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Figure  11.  Packaging,  Handling,  Storage  and  Transportation  (PHS&T)/ 
Life  Cycle  Relationship 


development  phase  special  requirements  concerning  packaging 
and  handling  are  evaluated  and  determined.  Protection 
against  environmental  and  transportation  damage  is  essential 
for  implementation  of  high  technology  automated  systems. 

Design  Interface .  The  design  interface  element  is  the 
relationship  of  logistics-related  design  parameters,  such  as 
Reliability  and  Maintainability  (R&M) ,  to  readiness  and 
logistics  support  resource  requirements  (ILS  Guide,  1986:7- 
12).  R&M  is  the  primary  design  interface  parameter  that 
will  be  evaluated  in  this  research.  Figure  12  illustrates 
the  design  interface  element  as  it  advances  through  the 
system  life  cycle  phases. 
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Figure  12.  Design  Interface/Life  Cycle  Relationship 


System  readiness  objectives  and  logistics  constraints 
are  established  during  the  conceptual  phase.  R&M,  both 
functions  of  the  design,  must  be  considered  during  the 
advanced  development  phase.  Reliability  can  be  defined  as 
"the  probability  that  a  system  will  perform  the  intended 
functions  for  a  predetermined  time  interval  in  a  planned 
environment"  (Ostrofsky,  1977:176).  It  is  a  quantitative 
measure  for  predicting  the  probability  of  failure  for  all  or 
part  of  a  given  system.  This  parameter  is  most  important  to 
the  supportability  of  a  system  when  it  is  designed  into  the 
system  early  in  the  life  cycle.  The  higher  the  reliability 
during  the  advanced  development  phase,  the  less  likelihood 
of  having  major  increases  in  support  requirements  later  in 
the  life  cycle  (Finkelstein  and  Guertin,  1988:52). 
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A  parameter  of  equal  significance  to  reliability  is 
n\3k  ainability.  Maintainability  is  defined  as  the 

"probability  that  a  failed  system  is  restored  to  operable  * 

condition  in  a  specified  downtime  in  its  intended 

environment"  (Ostrofsky,  1977:179).  Maintainability  must  be  * 

applied  early  in  the  design  process  to  prevent  high 
maintenance  costs  throughout  the  system  life  cycle 
(Finkelstein  and  Guertin,  1988:72). 

The  literature  review  substantiates  the  requirement  for 
determination  and  design  of  ILS  elements  early  in  the  system 
life  cycle  and  the  need  to  provide  ILS  updates  throughout 
the  system  life  cycle. 

Knowing  that,  on  the  average,  the  greatest  amount  of 
total  Life  Cycle  Cost  (LCC)  dollars  are  spent  on 
Operations  &  Maintenance  (O&M),  which  really  equates  to 
logistics  support,  and  that  the  vast  percentage  of 
total  LCC  is  committed  early  in  the  acquisition 
process,  it  becomes  apparent  that  logistics  planning 
must  begin  at  the  front  end  of  an  acquisition  program. 

(Materna  and  Andrews,  1988:5-4) 

Conclusion 

Automated  cartographic  or  photogrammetric  systems  can 
be  classified  as  high  technology,  fast  obsolescence  systems. 

Proper  management,  design,  and  support  of  these  systems  are 
required  to  maintain  mission  readiness.  Logisticians  must 
apply  ILS  techniques  during  the  entire  life  cycle  of 
automated  cartographic  or  photogrammetric  systems  especially 
in  the  conceptual,  advanced  development,  and  detailed  design 

V 

and  development  phases.  Coordination  between  logisticians 
and  designers-engineers  will  lower  support  costs  during  the 
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operational  use  phase  and  throughout  the  remaining  life 
cycle  of  the  system. 


Chapter  III  presents  the  methodology  used  in  this 
research.  This  literature  review  provides  the  basis  for 
developing  the  Delphi  survey  questions  concerning  logistics 
support  for  automated  cartographic  or  photogrammetric 
systems . 
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III.  Methodol oqy 


There  has  been  little  research  performed  concerning 
logistics  support  planning  for  automated  cartographic  or 
photogrammetric  systems.  As  one  of  the  largest  users  of 
these  specialized  systems,  the  Defense  Mapping  Agency  (DMA) 
sponsored  this  research  which  addressed  logistics  support 
planning  requirements  for  automated  cartographic  or 
photogrammetric  systems. 

The  research  design  executed  in  this  research  effort 
encompassed  three  major  phases.  The  first  phase  involved 
exploratory  research  to  include  a  detailed  literature  review 
concerning  the  areas  of  management  principles,  system  life 
cycle  or  design,  and  ILS.  The  second  phase  comprised  the 
construction  of  a  descriptive  ILS  planning  model  tailored  to 
the  DMA  operating  environment.  The  third  phase  validated 
the  descriptive  model  through  the  application  of  a  Delphi 
technique.  These  phases  provided  the  data  needed  to  answer 
the  research  questions  formulated  in  Chapter  I. 

Literature  Review 

The  literature  review  performed  in  Chapter  II 
identified  management  principles,  system  life  cycle  or 
design  processes,  and  ILS  elements  associated  with  automated 
cartographic  or  photogrammetric  systems.  These 
characteristics  and  attributes  facilitated  the  construction 
of  the  instrument  used  in  the  Delphi  survey. 
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Constructing  the  Model 

The  Delphi  method  provided  the  means  for  constructing  a 
descriptive  model  by  polling  expert  DMA  administrative, 
research  and  development,  support  and  maintenance,  and 
production  personnel  and  reaching  a  level  of  consensus  on 
characteristics  and  attributes  of  a  descriptive  model 
applicable  to  automated  cartographic  or  photogrammetric 
systems.  These  four  primary  functional  areas  formed  the  DMA 
expert  population  utilized  in  the  two  iterative  rounds  of 
the  Delphi  survey. 

Selection  of  Research  Technique .  A  particular  field  of 

research  follows  a  developmental  path  as  it  progresses 

through  the  maturation  cycle.  In  Figure  13,  the  arrow  shows 

the  directional  path  the  field  of  research  follows 

throughout  maturity  (Schendel  and  Cool,  1988:28). 

The  initial  implementation  of  research  begins  in  the 

prescriptive  cell  (see  Figure  13,  cell  1).  Schendel  and 

Cool  summarized  the  prescriptive  phase  as: 

...  the  worst  of  all  research  worlds  depicted;  where 
much  work  lies,  mostly  embodied  in  the  form  of 
untested,  or  worse,  untestable,  statements.  (Schendel 
and  Cool,  1988:28) 

The  descriptive  phase  of  research  methodology  attempts 
to  define  the  research  problem  (see  Figure  13,  cell  2).  The 
process  requires  personal  insight,  judgement,  and  creativity 
with  the  results  often  serving  as  hypotheses  which  require 
further  testing  (Schendel  and  Cool,  1988:28). 
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Sophistication  of  Research 


Immature  Mature 

Testing 

Hypothesis 

Generation  * 

Subjective  Objective 

Nature  of  Explanation 

Figure  13.  Maturation  of  Research  in  Scientific  Method 

(Clark,  1989:103) 

The  hypothesis  generation  phase  follows  the  subjective 
description  of  the  research  problem  (see  Figure  13,  cell  3). 

The  research  performed  in  this  phase  depicts  a  meticulous, 
accurate  description  of  the  research  problem  (Schendel  and 
Cool,  1988:28).  In  this  phase,  a  precise  description  of  the 
research  problem  often  leads  to  excellent  field  research  and 
complete  data  bases  required  to  analyze  the  research 
resul ts . 

Hypothesis  testing  is  the  final  phase  of  research 
methodology  and  the  most  optimal  stage  (see  Figure  13, 
cell  4).  Events  such  as  testing  hypotheses,  developing 
causal  models,  and  validating  predictive  theory  occur  during 
this  phase  (Schendel  and  Cool,  1988:29).  Using  statistical 
methods,  this  type  of  research  "can  guide  practice  and 
explain  results  achieved"  (Schendel  and  Cool,  1988:29). 

This  research  effort  lies  within  the  descriptive 
process  of  research  methodology.  Using  the  Delphi  technique 


Prescriptive 

Empirical 

Descriptive 
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to  solicit  DMA  experts'  opinions,  a  descriptive  model 
portraying  the  logistics  support  requirements  for  automated 
cartographic  or  photogrammetric  systems  provided  the 
boundaries  of  the  research  problem  area  (Schendel  and  Cool, 
1988:28).  The  descriptive  approach  defined  the  parameters 
and  allowed  for  further  research  in  the  logistics  support 
planning  area  of  automated  cartographic  or  photogrammetric 
systems . 

A  review  of  the  literature  determined  whether  the 

Delphi  technique  of  research  methodology  was  appropriate  for 

determining  logistics  support  requirements  for  automated 

cartographic  or  photogrammetric  systems. 

In  order  to  comprehensively  study  the  manifold  issues 
involved  in  the  application,  usefulness,  and  future 
applicability  of  the  Delphi  method,  it  was  necessary  to 
capture  the  thoughts  of  those  who  were  familiar  with 
the  method.  (Brockhaus  and  Mickelson,  1977:104) 

Brockhaus  and  Mickelson's  research  polled  312  individuals 

involved  with  the  Delphi  method.  Of  the  312  total 

respondents,  176  had  experience  with  the  Delphi  method  in 

the  capacity  of  a  project  director,  59  individuals 

participated  as  panel  members  in  Delphi  studies,  and  77 

individuals  possessed  familiarity  with  Delphi,  but  had  not 

been  involved  in  any  Delphi  studies  (Brockhaus  and 

Mickelson,  1977:106).  They  divided  their  findings  into  the 

following  five  sections: 

(1)  for  what  purposes  and  with  what  degree  of  success 
has  the  Delphi  method  been  used;  (2)  how  significant  a 
development  has  the  Delphi  method  been  and  how 
frequently  is  it  utilized  throughout  the  world  in 
various  substantive  fields;  (3)  what  will  be  the  future 
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of  the  Delphi  method;  (4)  how  should  the  Delphi  method 
be  used  in  conjunction  with  other  methodologies;  (5) 
what  types  of  organizations  have  financed  Delphi 
studies;  and  (6)  other  considerations  affecting  the 
applicability  of  the  Delphi  method.  (Brockhaus  and 
Mickelson,  1977:105) 

The  first  finding  indicated  the  Delphi  method  has  been 

f 

successful  when  used  for  planning  purposes  associated  with 
technological  applications  (Brockhaus  and  Mickelson, 

1977:106).  This  type  of  application  supports  the  selection 
and  use  of  the  Delphi  method  for  determining  logistical 
support  requirements  for  automated  cartographic  or 
photogrammetric  systems.  When  fully  operational  in  the  mid 
1990s,  DMA's  automated  cartographic  or  photogrammetric 
systems  will  provide  state-of-the-art  graphic  and  digital 
earth  data  products  (DOD/DMA  DPS  Handbook,  1989:1). 

Brockhaus  and  Mickelson' s  research  revealed  five 
substantive  fields  of  application  for  Delphi  studies 
(Brockhaus  and  Mickelson,  1977:106).  Substantive  fields, 
where  the  Delphi  method  has  been  used,  are  illustrated  in 
Table  3.  Logistics  support  requirements  for  automated 
cartographic  or  photogrammetric  systems  fall  within  the 
physical  sciences/engineering  field  application  of  the 
Delphi  method.  Project  directors  who  conducted  two  or  more 
studies  in  the  physical  sciences/engineering  field  found  the 

* 

Delphi  method  to  be  a  significant  development  within 
scientific/engineering  research  (Brockhaus  and  Mickelson, 

1977:107). 
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Table  3 


Number  of  Delphi  Studies  by  Substantive  Field 


Substantive  field 

Number 

of 

studies 

reported 

Percentage 
of  total 
studies 
reported 

Physical  sciences/engineering 

153 

26% 

Business/ economics 

139 

23% 

Social  sciences 

140 

23% 

Education/public  administration 

116 

19% 

Biological  sciences/medicine 

50 

09% 

Total 

598 

100% 

(Brockhaus 

and  Mickelson 

.  1977:106) 

Forecasts  revealed  a  moderate  growth  for  Delphi  studies 

in  the  physical  sciences/ engineering  and  business/economics 

fields. and  substantial  growth  for  the  social  sciences  and 

education  fields  (Brockhaus  and  Mickelson,  1977:107). 

Brockhaus  and  Mickelson's  research  indicated  certain 

methodologies  should  be  applied  prior  to  and  after  the 

implementation  of  the  Delphi  method.  Brockhaus  and 

Mickelson  affirmed  the  need  for  both  the 

. . .  development  of  an  information  database  through  a 
literature  review  prior  to  the  Delphi  application  and 
model  building  following  the  Delphi  survey.  (Brockhaus 
and  Mickelson,  1977:108) 

The  literature  review  conducted  prior  to  the  Delphi  survey 
in  this  research  effort  provided  an  information  database 
containing  management  principles,  system  life  cycle  or 
design  processes,  and  ILS  elements.  The  information 
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collected  in  the  literature  review  provided  the  necessary 
data  required  to  construct  the  Delphi  survey.  Following  the 
Delphi  survey,  a  descriptive  ILS  planning  model  was 
constructed  and  tailored  to  DMA  automated  cartographic  or 
photogrammetric  systems. 

The  cost  and/or  available  funds  to  conduct  this  Delphi 
study  had  no  bearing  on  the  selection  of  the  technique  used 
for  determining  logistical  support  requirements  for 
automated  cartographic  or  photogrammetric  systems.  DMA 
identified  the  need  for  logistics  management  research  in  the 
cartography  ol  photogrammetry  field  and  funded  the  research 
through  its  Long-Term  Full  Time  Training  Program. 

Brockhaus  and  Mickelson's  research  indicated  most 
Delphi  studies  consist  of  three  or  less  iterations 
(Brockhaus  and  Mickelson,  1977:109).  The  Delphi  study 
associated  with  this  research  was  limited  to  two  iterations 
because  of  time  limitations  placed  on  the  thesis  effort. 

The  major  reason  for  selecting  the  Delphi  method  was  to 
gather  expert  opinion  and  bring  the  expected  diversity  of 
opinions  to  a  convergence  (Brockhaus  and  Mickelson, 
1977:109).  Although  faced  with  time  constraints,  the  Delphi 
method  allowed  for  the  participation  of  physically  dispersed 
experts  while  maintaining  a  high  degree  of  convergence 
(Preble,  1983:76).  "In  using  the  Delphi  approach,  numerous 
experts  are  solicited  for  their  opinions  on  the  future 
technology  in  a  specified  area"  (Roberts,  1969-1970:113). 

The  Delphi  method,  which  polls  DMA  experts  for  their 


opinions,  attempted  to  reach  a  level  of  consensus  on 
logistical  support  requirements  needed  to  support  and 
maintain  technologically  advanced  automated  cartographic  or 
Fhotogrammetric  systems.  The  normative  use  of  the  Delphi 
*  technique  identifies  and  orders  the  characteristics  and 

attributes  of  the  model  developed  (Preble,  1983:83). 

Delphi  Method.  Jillson  stated  that  the  typical  Delphi 
researcher  is  "The  graduate  student  who  worries  that  his 
thesis  proposal  seems  a  bit  dull,  and  believes  that  there  is 
no  dissertation  like  a  spiffy  dissertation"  (Jillson, 
1975:221).  The  following  discussion  of  the  Delphi  method 
refutes  Jil Ison’s  notion  of  the  typical  Delphi  user  and 
explains  why  Delphi  was  applied  to  this  research. 

The  Delphi  technique  employs  a  questionnaire  for 
organizing  and  sharing  opinion  through  feedback  (Bardecki, 
1984:281).  The  technique  usually  is  accomplished  through  an 
iterative  process  in  which  experts  are  solicited  for  their 
opinions  concerning  the  topic  of  research.  Figure  14 
illustrates  the  Delphi  technique  in  closer  detail. 

The  Delphi  technique,  as  any  other  methodology,  begins 
with  problem  definition.  Having  defined  the  research 
problem,  the  appropriate  degree  of  expertise  is  determined 
%  for  the  Delphi  survey  respondents.  Experts  are  selected 

possessing  the  knowledge  to  adequately  provide  opinion  data 
-  and  numerical  feedback  to  survey  questions.  While  preparing 

the  survey  questionnaire  to  address  investigative  research 
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START 

Problem  definition 

Determine  expertise 
required 

,  I 

Select  experts 
(sample  size) 

Prepare  questionnaire 

Distribute  questionnaire  < 

,  I 

Analyze  questionnaire 
responses 

-  yes  Has  consensus  been  reached? 

no 

Provide  requested  information 
and  tabulate  responses 

Prepare  the  next  questionnaire 

- >  Compile  final  responses 

and  disseminate  results  (final  report) 

Figure  14.  Delphi  Process  Flowchart 

(Riggs,  1983:90) 

questions,  the  participation  of  each  selected  respondent  is 
confirmed  by  oral  or  written  correspondence.  Distributed 
questionnaires  require  thorough,  accurate,  and  timely 
responses.  Results  from  the  first  round  of  the  survey  are 
analyzed  to  determine  if  consensus  has  been  reached.  Non¬ 
consensus  items  and  all  opinion  data  from  previous  rounds  of 
the  survey  are  provided  as  feedback  to  the  participants  in 
successive  rounds  of  the  survey.  "The  process  is  continued 
in  an  iterative  manner  until  consensus  is  reached”  (Riggs, 
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1983:90).  Final  results  of  the  survey  should  be  provided  to 
all  Delphi  participants. 

Advantages  of  Delphi . 

The  Delphi  technique  was  developed  at  the  RAND 
Corporation  in  1950  by  Dalkey  and  his  associates  to 
eliminate  many  of  the  negative  effects  related  to  the 
use  of  interacting  groups  for  decision  making.  (Riggs, 
1983:89) 

Some  of  these  negative  effects  of  large  group  decision 
making  were  factors  such  as  expense,  conformity,  and 
dominance  of  certain  personnel.  The  Delphi  process  overcame 
the  negative  effects  of  group  interaction  decision  making 
and  provided  features  such  as  structured  format,  systematic 
procedures,  clear  communication,  statistical  feedback,  and 
anonymity  (Preble,  1983:75).  While  the  Delphi  technique 
exhibited  several  strengths,  it  also  exhibited  a  number  of 
distinct  weaknesses. 

Disadvantages  of  Delphi .  The  two  primary  disadvantages 
of  Delphi  applicable  to  this  research,  as  in  many  other 
Delphi  studies,  were  length  of  time  and  respondent 
selection.  Delphi  studies  are  extremely  time  consuming 
depending  on  the  number  of  iterations  performed  and  the 
amount  of  opinion  data  that  must  be  collated  and  distributed 
to  the  respondents  at  each  iteration.  The  time  limitation 
of  the  Air  Force  Institute  of  Technology  (AFIT)  graduate 
logistics  management  program  permitted  only  two  iterations 
of  the  Delphi  survey. 

The  opinion  data  collected  in  round  one  of  the  survey 
and  distributed  as  feedback  in  round  two  of  the  survey 
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allowed  respondents  to  consider  data  provided  by  other 
experts.  In  round  two  of  the  survey,  respondents  refuted 
opinions  of  others  and  provided  additional  comments  to 
questions . 

Population  definition  can  be  achieved  relatively 
easily,  but  the  actual  respondent  selection  is  difficult. 
Iterative  interviews  of  senior  DMA  managers  provided  the  DMA 
Delphi  respondent  group.  This  process  is  explained  in  more 
detail  below  on  page  51.  DMA  experts  selected  for  the 
Delphi  survey  provided  accurate  and  timely  responses  in  both 
rounds  of  the  survey.  A  report  of  the  research  findings 
offered  an  incentive  to  respond  because  time  was  a  valuable 
asset  to  the  DMA  experts.  The  methodological  value  of  the 
Delphi  process  required  examining  the  reliability  and 
validity  of  the  technique. 

Rel iabi 1 itv .  "Reliability  is  concerned  with  estimates 

of  the  degree  to  which  a  measurement  is  free  of  random  or 

unstable  error"  (Emory,  1985:98).  In  a  discussion  of  Delphi 

method  reliability  Hill  and  Fowles  asserted 

...  the  reliability  of  Delphi  forecasting  is  critically 
weakened  by  an  absence  of  recognized  administrative 
standards  to  guide  Delphi  implementation,  along  with  an 
insufficiency  of  the  methodological  expertise  which 
would  be  necessary  to  make  up  for  the  first  deficit 
when  a  Delphi  is  actually  executed.  (Hill  and  Fowles, 
1975:184) 

Jillson  supported  this  premise  in  her  article  "Developing 
Guidelines  for  the  Delphi  Method".  She  stated 
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...  I  would  suggest  that  the  time  has  come  for 
guidelines  or  standards  to  be  developed  that  can  be 
applied  to  the  review  of  Delphi  proposals,  or  reports 
of  findings  resulting  from  Delphi  studies.  (Jillson, 
1975:222) 

Jillson  recommended  that  these  guidelines  include  some  of 
♦  the  following: 

Standards  for  determining  the  applicability  of  the 
technique  to  the  problem  identified;  criteria  for 
selecting  respondents;  and  recommendations  for  the 
interpretation  f  results.  (Jillson,  1975:222) 

Guidelines  for  determining  the  applicability  of  Delphi  to 

the  research  problem  were  established  earlier  in  this 

chapter.  Iterative  interviews  of  senior  DMA  managers 

provided  criteria  and  names  of  appropriate  DMA 

administrative,  research  and  development,  support  and 

maintenance  and  production  respondents  (see  page  51). 

Recommendations  for  the  interpretation  of  research  results 

were  of  a  planning  nature.  The  descriptive  model  developed 

from  the  survey  served  as  a  logistics  support  planning  tool 

for  automated  cartographic  or  photogrammetric  systems. 

Adherence  to  the  guidelines  proposed  by  Jillson  ensured  the 

reliability  of  the  Delphi  method  employed  in  this  research. 

Validity .  T’.e  validity  of  the  Delphi  method  can  be 

examined  in  two  ways:  (1)  data  validity;  and  (2)  internal 

validity  (Hill  and  Fowles,  1975:185). 

Data  validity  refers  to  the  data  that  result  from 

conducting  the  Delphi  survey  (Hill  and  Fowles,  1975:185). 

The  selection  of  DMA  experts  and  the  qualitative  and 

quantitative  feedback  from  the  experts  provided  the 
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necessary  data  to  accurately  determine  logistical  support 
requirements  for  automated  cartographic  or  photogrammetric 
systems.  The  use  of  DMA  experts  in  these  automated  systems 
ensured  sufficient  content  validity  in  the  research  in  that 
it  provided  adequate  coverage  of  the  topic  under  study 
(Emory,  1985:95). 

Internal  validity  is  "the  ability  of  a  research 
instrument  to  measure  what  it  is  purported  to  measure" 
(Emory,  1985:94).  The  literature  provided  both  negative  and 
positive  viewpoints  concerning  the  internal  validity  of  the 
Delphi  technique.  Hill  and  Fowles  expressed  serious 
reservations  regarding  the  achievability  of  internal 
validity  through  the  use  of  the  Delphi  method  as  shown 
below. 

. i .  neither  data  nor  internal  validity  have  yet  been 
demonstrated.  Until  they  are,  it  is  reasonable  to 
remain  skeptical,  for  certain  characteristics  of  the 
method  call  any  claims  for  validity  into  question. 

(Hill  and  Fowles,  1975:186) 

In  support  of  the  internal  validity  of  Delphi,  Martino 
stated  "there  is  a  well-defined  process  at  work  in  the 
production  of  Delphi  forecasts"  (Martino,  1972:299).  Hills 
and  Fowles  also  provided  evidence  in  support  of  internal 
validity  by  stating 

. . .  other  evidence  advanced  m  support  of  internal 
validity  is  that  from  Dalkey's  research  employing 
students  to  "predict"  almanac-type  information  (current 
detailed  facts)  or  Campbell's  U3e  of  Delphi  for  short¬ 
term  estimates  of  the  performance  of  economic 
indicators.  (Hill  and  Fowles,  1975:186) 
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Population  Definition.  The  population  of  the  Delphi 
survey  respondents  comprised  all  employees  of  the  Defense 
Mapping  Agency  associated  with  administrative,  research  and 
development,  support  and  maintenance,  and  production 
activities  for  automated  cartographic  or  photogrammetric 
systems.  Appendix  D  illustrates  the  DMA  organizational 
structure  and  the  various  office  functions  associated  with 
administrative,  research  and  development,  support  and 
maintenance,  and  production  activities  for  automated 
cartographic  or  photogrammetric  systems. 

Delphi  Respondents .  Interviews  were  held  with  the 
Director,  the  Deputy  Director/Operations  Support  Group,  and 
the  Deputy  Director/Modernization  Development  Group  of  the 
DMA  Systems  Center  to  determine  an  initial  list  of  possible 
Delphi . respondents  within  the  office  functions  identified  in 
Appendix  D.  An  iterative  phone  call  inquiry  process, 
starting  with  the  initial  list  of  possible  Delphi 
respondents,  led  to  the  final  selection  of  Delphi 
respondents  for  the  survey. 

Thirty-five  DMA  experts  from  administrative,  research 
and  development,  support  and  maintenance,  and  production 
functions  were  selected  from  the  population  mentioned  above. 
Thirty-one  experts  (89%  response  rate)  participated  in  the 
Delphi  survey.  Table  4  illustrates  the  composition  of  the 
Delphi  respondent  group.  The  four  functional  areas  selected 
are  portrayed  along  the  vertical  axis  of  Table  4.  These 
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Table  4 


Delphi  Respondents  by  Function  and  Office  Symbol 


Function 

EMAHO 

DMASC  EMARC 

EMAHTC 

EMAAC 

EMATSC 

Total 

Admin. 

2 

2 

1 

1 

1 

7 

Research 

and 

Development 

2 

4 

1 

1 

1 

9 

Support 

and 

Maintenance 

1 

6 

7 

Production 

1 

2 

3 

2 

8 

Total 

5 

13 

4 

5 

3 

1 

31 

functional 

areas 

comprise 

the 

sources 

of  experts  within  DMA 

most  appropriate  to  participation  in  this  survey.  Every 
effort  was  made  to  attain  an  equal  representation  from  the 
four  functional  respondent  groups.  The  horizontal  axis 
depicts  the  six  major  office  components  within  DMA.  With 
the  exception  of  DMA  Systems  Center  (DMASC)  and  DMA 
Telecommunications  Services  Center  (DMATSC) ,  there  was  equal 
respondent  representation.  DMASC  responsibilities  include 
design,  development,  and  maintenance  for  DMA  automated 
cartographic  or  photogrammetric  systems  explaining  the  large 
representations  in  these  two  functional  areas.  The  primary 
role  of  DMATSC  is  coordination  of  all  telecommunications 
activities  for  DMA,  rarely  interrelated  with  cartographic  or 
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photogrammetric  applications  which  explains  the  low 
representation  of  this  office  in  this  research  effort.  The 
following  descriptions  provide  the  functional 
responsibilities  of  the  six  office  components  represented  in 
the  Delphi  survey: 

DMA  Headquarters  (DMAHQ)  provides  high-level  policy 
guidance  and  oversight  to  all  components  concerning 
administrative,  research  and  development,  support  and 
maintenance,  and  production.  (DOD/DMA  DPS  Handbook, 
1989:59) 

The  DMA  Systems  Center  (DMASC)  is  responsible  for  the 
requirements  definition,  design,  development, 
procurement,  installation,  transition,  and  maintenance 
of  the  Digital  Production  System  as  well  as  all  other 
research  and  development  activities,  production  line 
maintenance,  and  configuration  management.  (DOD/DMA  DPS 
Handbook,  1989:59) 

The  three  DMA  production  centers,  DMA  Reston  Center 
(DMARC) ,  DMA  Aerospace  Center  (DMAAC),  and  DMA 
Hydrographic  Topographic  Center  (DMAHTC) ,  support  the 
Digital  Production  System  development  and  transition 
activities  primarily  through  two  organizations: 

(1)  Research  and  Engineering  Directorate  (RE)  and 

(2)  Production  Programs  Modernization  Division  (PPM). 
(DOD/DMA  DPS  Handbook,  1989:60) 

DMA  Telecommunications  Services  Center  (DMATSC)  is 
responsible  for  the  overall  coordination  of 
telecommunications  activities  across  DMA  and  for 
operation  and  maintenance  of  the  Integrated  DMA 
Telecommunication  System  (IDTS).  (DOD/DMA  DPS  Handbook, 
1989:61) 

The  response  base  was  supportive  of  the  logistics 
support  planning  element  for  DMA.  The  Delphi  survey 
solicited  opinions  from  the  four  functional  areas  depicted 
in  Table  4.  Administrative  functions  for  the  offices 
identified  in  Appendix  D  participated  in  the  survey  with  two 
notable  exceptions.  The  Director  of  DMA  retired  from 
military  service  during  the  conduct  of  the  first  round  of 


the  survey.  The  Director  of  DMA  Aerospace  Center  (DMAAC) 
did  not  feel  qualified  to  adequately  address  the  questions 
in  the  survey  because  of  his  limited  background  in 
cartographic  or  logistics  applications.  There  was  only  one 
research  and  development  representative  from  DMA  Reston 
Center  ( DMARC )  because  of  the  limited  number  of  personnel 
available  at  the  time  of  the  study.  DMA  Telecommunications 
Services  Center  did  not  staff  a  research  and  development 
function;  therefore,  it  had  no  functional  representation  for 
research  and  development.  The  Director  of  DMAHQ  Facilities 
Engineering  &  Logistics  Directorate  and  the  Deputy  Director 
of  DMA SC  Operations  Support  Group  did  not  participate 
because  they  were  preoccupied  with  organizational  changes  at 
the  time  of  the  survey. 

Support  and  maintenance  functions  were  primarily 
located  at  DMA  Headquarters  (DMAHQ)  and  DMA  Systems  Center 
(DMASC)  and  the  participant  selections  supported  this  fact. 
The  production  functions  within  DMA  were  located  at  the  DMA 
Hydrographic/Topographic  Center  (DMAHTC) ,  the  DMA  Aerospace 
Center  (DMAAC),  and  the  DMARC.  These  three  offices  were 
equally  represented  in  the  survey.  One  respondent  was  from 
DMASC,  but  has  worked  in  his  present  position  less  than  one 
year.  He  was  selected  based  on  his  extensive  experience  in 
a  production  element  gained  elsewhere  in  DMA.  Each 
respondent  selected  was  contacted  by  phone  and  written 
correspondence  to  confirm  participation  prior  to  mailing  the 


surveys . 


Delphi  Survey  Construction.  Delphi  survey  questions 
were  developed  from  the  data  researched  in  the  literature 
review.  The  survey,  found  in  Appendix  E,  contained  four 
sections  pertaining  to  (1)  personnel  demographics;  (2)  basic 
♦  characteristics  of  the  Logistics  Systems  Management  Matrix 

(LSMM) ;  (3)  unique  factors  of  ILS  elements  associated  with 
automated  cartographic  or  photogrammetric  systems;  and 
(4)  life  cycle  determination  and  design  requirements  of  the 
ILS  elements.  Attachments  2  and  3  of  the  survey  provided 
detailed  descriptions  of  the  design  or  life  cycle  phases  and 
the  ILS  elements  portrayed  in  the  LSMM.  The  definitions 
provided  a  common  basis  of  understanding  for  the  respondents 
to  ensure  non-biased  results  from  the  questionnaire. 

The  survey  consisted  of  multiple  choice  questions, 
scaled. questions  (Likert),  rank  order  questions,  and  open 
ended  questions  that  allowed  the  experts  to  provide  opinions 
concerning  logistical  characteristics  and  attributes  of 
automated  cartographic  or  photogrammetric  systems.  A  model 
was  constructed  from  the  responses  received  from  DMA 
administrative,  research  and  development,  support  and 
maintenance,  and  production  experts. 

The  survey  was  pretested  by  four  DMA  experts  within 
'  each  of  the  functional  areas.  The  responses  from  the 

pretest  surveys  aided  in  clarifying  questions  and  providing 
.  attachments  to  assist  the  Delphi  respondents  in  survey 

completion . 
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Data  Analysis .  Using  the  Delphi  method,  one  continues 
the  process  in  an  iterative  manner  until  consensus  is 
reached  (Riggs,  1983:90).  Due  to  the  limited  time  available 
to  conduct  the  Delphi  process  for  this  research  effort,  a  60 
percent  agreement  level  was  established  as  the  consensus 
criterion  (Gregor,  1988:40).  Scaled,  multiple  choice,  rank- 
ordered,  and  open-ended  responses  provided  by  the  experts 
were  subjected  to  the  consensus  criterion. 

Mean  responses  were  computed  for  the  scaled  questions. 
For  Likert  scale  items  requesting  a  scaled  response 
"expressing  either  a  favorable  or  unfavorable  attitude" 
(Emory,  1985:255),  "highly  agree/agree"  and  "highly 
disagree/disagree"  responses  were  grouped  together  in  order 
to  determine  whether  consensus  was  reached  (Gregor, 

1988:41).  The  other  Likert  scale  item  responses  were  not 
grouped  together,  but  still  were  subjected  to  the  60  percent 
consensus  rule. 

For  the  non-Likert  questions,  frequencies  were  examined 
to  determine  if  the  60  percent  consensus  existed.  Means 
were  computed  for  the  items  of  one  rank-ordered  response. 

The  items  were  ranked  according  to  the  means  and  also 
subjected  to  the  60  percent  consensus  rule.  For  the  other 
rank-ordered  responses  a  weighting  factor  was  assigned  to 
each  response  to  determine  the  three  MOST  and  LEAST 
important  ILS  elements  associated  with  logistics  support 
planning  of  automated  cartographic  or  photogrammetric 
systems.  The  items  were  ranked  according  to  the  total  value 
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computed  for  each  ILS  element  and  also  subjected  to  the  60 
percent  consensus  rule. 

A  list  of  factors,  unique  to  automated  cartographic  or 
photogrammetric  systems,  was  developed  for  each  ILS  element 
of  the  model  in  round  one  of  the  survey.  In  round  two,  the 
experts  selected  the  three  most  important  factors  associated 
with  automated  cartographic  or  photogrammetric  systems. 

Other  comments  included  in  round  one  were  provided  as 
feedback  in  round  two  to  assist  the  experts  in  answering 
non-consensus  items.  Only  those  questions  with  less  than  60 
percent  consensus  agreement  were  asked  again  in  round  two. 
However,  mean  and  percent  of  consensus  agreement  for  all 
questions  were  provided  as  feedback. 

Validating  the  Model 

The  model  constructed  was  validated  by  using  the 
iterative  Delphi  process.  The  survey  process  determined  if 
DMA  management  believed  the  characteristics  and  attributes 
of  the  model  were  valid  for  DMA's  high-technology,  fast- 
obsolescence  systems.  In  round  one  of  the  survey,  the  DMA 
experts  reached  consensus  on  the  three  dimensions  of  the 
descriptive  model.  The  opinion  data  also  supported  the 
dimensions  of  the  model.  The  descriptive  model  validated  by 
DMA  experts  was  identical  to  Ostrofsky’s  Logistics  Systems 
Management  Matrix  (LSMM)  presented  in  Chapter  II  with  one 
exception.  Security  was  added  to  the  ILS  elements  dimension 
of  the  model  from  opinion  responses  provided  by  the  experts. 
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Conclusion 


The  Defense  Mapping  Agency  recognized  the  need  to 
identify  logistics  support  planning  requirements  for  * 

automated  cartographic  or  photogrammetric  systems.  In  an 
area  of  research  defined  as  prescriptive,  at  best,  it  was  * 

imperative  to  establish  parameters  for  investigating  the 
research  problem.  The  Delphi  method  provided  the  technique 
required  to  develop  a  descriptive  model  portraying  the 
logistics  support  requirements  for  automated  cartographic  or 
photogrammetric  systems.  Two  iterations  of  the  Delphi 
survey  were  performed  using  DMA  experts  employed  in 
administrative,  research  and  development,  support  and 
maintenance,  and  production  functional  activities.  The 
results  of  the  Delphi  process  are  found  in  the  following 
chapter. 
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IV .  Results 

The  results  chapter  describes  the  results  obtained  from 
the  Delphi  survey  conducted  in  this  research.  The  Delphi 
survey  of  35  DMA  experts  (administrative,  research  and 
development,  production,  operations  support  and  maintenance) 
consisted  of  two  iterative  rounds  of  questions  to  determine 
(1)  if  DMA  experts  could  reach  a  level  of  consensus  on  the 
characteristics  and  attributes  of  a  logistics  support  model 
applicable  to  automated  cartographic  or  photogrammetric 
systems;  (2)  what  factors  within  the  ILS  elements  were 
unique  and  applicable  to  automated  cartographic  or 
photogrammetric  systems;  and  (3)  at  what  stages  of  the 
automated  system's  life  cycle  did  the  ILS  elements  require 
determination  and  design?  Based  on  the  Delphi  survey 
results,  a  descriptive  model  was  developed  to  address  the 
logistical  support  requirements  for  automated  cartographic 
or  photogrammetric  systems. 

Delphi  Survey 

The  purpose  of  the  Delphi  survey  was  to  verify  the 
management  principles,  system  life  cycle  or  design 
processes,  and  ILS  elements  used  in  Ostrofsky's  Logistics 
Systems  Management  Matrix  (LSMM)  and  determine  what 
characteristics  or  attributes  associated  with  automated 
cartographic  or  photogrammetric  systems  should  be  included 
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in  the  model .  The  survey  consisted  of  two  separate 
iterative  rounds  of  questioning. 

The  round  one  questionnaire  (Appendix  E)  contained  the  4 

following  four  topic  areas  of  consideration  (1)  background 
information;  (2)  basic  characteristics  of  the  LSMM  model;  * 

(3)  unique  factors  of  ILS  elements  associated  with  automated 
cartographic  or  photogrammetric  systems;  and  (4)  life  cycle 
determination  and  design  requirements  of  the  ILS  elements. 

Round  one  of  the  survey  consisted  of  19  multiple  choice 
questions,  six  scaled  questions  (Likert),  two  rank  order 
questions,  and  17  open  ended  questions  requesting  the 
experts  to  specify  logistical  characteristics  and  attributes 
of  automated  cartographic  or  photogrammetric  systems. 

Responses  from  some  open  ended  questions  provided  in  round 
one  served  as  input  for  constructing  follow-on  questions 
used  in  round  two  of  the  Delphi  survey.  The  Delphi 
respondents  also  included  many  additional  comments 
throughout  the  survey.  Those  comments  provided  in  round  one 
of  the  survey  are  found  in  Appendix  F. 

The  round  two  questionnaire  (Appendix  G)  contained 
three  of  the  four  topic  areas  which  were  found  in  round  one 
of  the  Delphi  survey.  The  topic  area  of  background 

information,  which  contained  demographic  data  on  the  survey  * 

participants,  was  not  repeated  in  round  two.  Not  all 
questions  required  answers  in  the  second  round  because 
consensus  was  reached  in  the  first  round  of  the  Delphi 
survey.  As  stated  in  Chapter  III,  the  criterion  for 
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consensus  was  established  at  60%  agreement  on  a  single 
response  item.  The  percent  of  consensus  agreement  was 
provided  for  each  answer. 

The  questions  which  required  a  new  response  in  round 
two  were  asterisked  on  the  survey  questionnaire  and 
accompanied  by  the  statement  "Your  New  Response"  (see 
Appendix  G).  For  multiple  choice,  scaled,  and  rank  order 
questions,  mean  ratings  and/or  comments  from  round  one  were 
provided  to  assist  the  respondent  in  completing  non¬ 
consensus  questions.  For  each  ILS  element  a  list  of  factors 
unique  to  automated  cartographic  or  photogrammetric  systems 
was  collated  from  round  one  open-ended  responses.  Each 
participant  reviewed  the  list  and  provided  the  three  most 
important  factors,  for  each  ILS  element,  that  he/she  felt 
were  unique  to  automated  cartographic  or  photogrammetric 
systems.  Additional  comments  provided  in  round  two  of  the 
survey  are  provided  in  Appendix  H. 

Round  One  Resul ts 

The  response  rate  from  round  one  of  the  Delphi  survey 
was  89%  with  participation  from  31  of  35  DMA  administrative, 
research  and  development,  production,  operations  support  and 
maintenance  experts.  Appendix  I  lists  the  respondents  by 
name  and  organizational  component.  The  numerical  responses 
and  open-ended  comments  provided  by  the  experts  were  of 
great  value  in  the  research  effort.  Round  one  of  the  survey 
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was  divided  into  four  topic  areas  as  mentioned  above.  The 
experts'  responses  to  these  topic  areas  are  examined  below. 

Background  Information .  For  the  total  respondents 
(N  =  31),  the  functional  responsibilities  of  the  experts 
were  evenly  distributed.  Table  5  illustrates  that  seven  DMA 
administrative,  nine  research  and  development,  eight 
production,  and  seven  operations  support  and  maintenance 
experts  took  part  in  round  one  of  the  Delphi  survey. 

Table  5 

Functional  Roles  of  Delphi  Experts  --  Round  One  Delphi  Survey 


Code 

Cateqorv 

Frequency 

1 

Administrative 

7 

2 

Research  and  Development 

9 

3 

Operational  Use  (Production) 

8 

4 

-  Support  and/or  Maintenance 

7 

TOTAL 

31 

Table  6  depicts  the  organizational  representation  of 
the  Delphi  respondents.  The  largest  organizational  response 
was  from  the  Defense  Mapping  Agency  Systems  Center  (DMASC) . 
Many  of  the  Delphi  respondents  are  assigned  to  DMASC  where 
there  are  large  research  and  development,  and  operations 
support  and  maintenance  functional  responsibilities. 

Table  7  illustrates  that  the  dominant  age  group  of  the 
Delphi  respondents  was  between  41  and  50  years.  All  31 
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Table  6 


Organizational  Representation  of  Delphi  Experts  -- 
Round  One  Delphi  Survey 


« 


Code 

Cateaorv 

Frequency 

1 

DMA  Headquarters 

5 

2 

DMA  Systems  Center 

13 

3 

DMA  Reston  Center 

4 

4 

DMA  Hydrographic/Topographic  Center 

5 

5 

DMA  Aerospace  Center 

3 

6 

Other  (Telecommunication  Services  Center) 

1 

TOTAL 

31 

Table  7 

Age  of  Delphi  Experts  --  Round  One  Delphi 

Survey 

Code 

Cateqorv 

Frequency 

1 

Less  than  30  years 

0 

2 

31  to  40  years 

10 

3 

41  to  50  years 

17 

4 

51  to  60  years 

4 

5 

More  than  60  years 

0 

TOTAL 

31 

respondents  were  over  the  age  of  30  and  21  (67.7%)  of  the 
respondents  were  over  the  age  of  40. 

Table  8  illustrates  that  all  of  the  DMA  experts  who 
responded  had  at  least  a  bachelor's  degree  with  74%  having  a 
graduate  degree  of  higher  level  of  education. 
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Table  8 


Education  Level  of  Delphi  Experts  --  Round  One  Delphi  Survey 


Code  Category 


Frequency 


1  High  school  graduate  or  GED 

2  Some  college  work 

3  Associate  degree 

4  Bachelors  degree 

5  Some  graduate  work 

6  Master's  degree 

7  Master's  degree  plus 

8  Ph.  D.  or  equivalent 


0 

0 

3 

5 

8 

13 

2 


TOTAL 


31 


Table  9  illustrates  how  long  the  Delphi  experts  have 
been  associated  with  automated  cartographic  or 
photogrammetric  systems.  Nearly  40%  of  the  respondents  had 
20  or  more  years  of  experience  with  DMA  related  systems. 

Table  9 

Automated  Cartographic  or  Photogrammetric  Experience 
of  Delphi  Experts  --  Round  One  Delphi  Survey 


Code  Category 


Frequency 


1  Less  than  4  years 

2  4  years,  but  less  than  8  years 

3  8  years,  but  less  than  12  years 

4  12  years,  but  less  than  16  years 

5  16  years,  but  less  than  20  years 

6  20  years  or  more 


5 

4 

4 

4- 

2 

12 


TOTAL 


31 
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Table  10  illustrates  the  number  of  Delphi  experts  with 
experience  as  a  user  of  DMA  automated  cartographic  or 
photogrammetric  systems  outside  of  DMA.  Only  four 
respondents  (13%)  indicated  experience  with  DMA  automated 
cartographic  or  photogrammetric  systems  outside  of  DMA. 
Nearly  all  the  Delphi  experts'  experience  with  automated 
cartographic  or  photogrammetric  systems  has  been  acquired 
while  employed  with  DMA. 


Table  10 

Delphi  Experts'  Experience  as  User  of  DMA  Automated 
Cartographic  or  Photogrammetric  Systems  Outside  of  DMA 
--  Round  One  Delphi  Survey 


Code 

Category 

Frequency 

1 

Yes 

4 

2 

No 

27 

TOTAL 

31 

Table  11  illustrates  the  length  of  time  the  Delphi 
experts  have  been  employed  at  DMA.  The  frequency 
distribution  in  this  table  is  nearly  identical  to  the  ranges 
of  experience  in  automated  cartographic  or  photogrammetric 
systems  reported  in  Table  9. 

Table  12  illustrate?  how  long  the  Delphi  experts  have 
worked  in  their  present  position.  Nearly  52%  of  the  experts 
worked  in  their  present  position  less  than  two  years,  while 
90%  worked  in  their  present  position  less  than  four  years. 
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Table  11 


Length  of  DMA  Employment  of  Delphi  Experts 
--  Round  One  Delphi  Survey 


Code 

Cateaorv 

Freauencv 

1 

Less  than  4  years 

6 

2 

4  years,  but  less  than  8  years 

4 

3 

8  years,  but  less  than  12  years 

5 

4 

12  years,  but  less  than  16  years 

2 

5 

16  years,  but  less  than  2C  years 

2 

6 

20  years  or  more 

12 

TOTAL 

Table  12 

Length  of  Time  in  Present  Position  of  Delphi 
--  Round  One  Delphi  Survey 

Experts 

Code 

Category 

Frequency 

1 

Less  than  1  year 

8 

2 

1  year,  but  less  than  2  years 

8 

3 

2  years,  but  less  than  3  years 

6 

4 

3  years,  but  less  than  4  years 

6 

5 

4  years,  but  less  than  5  years 

2 

6 

5  years  or  more 

1 

TOTAL 

31 

* 


Table  13  illustrates  the  various  DMA  organizations  in 
which  the  Delphi  experts  had  experience.  There  were  eleven 
respondents  who  indicated  they  had  experience  in  more  than 
one  of  the  organizations  within  DMA.  There  were  only  three 
respondents  who  indicated  they  had  no  other  organizational 
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able  13 


Organizational  Experience  Within  DMA  of  Delphi  Experts 
--  Round  One  Delphi  Survey 


Code 

Category 

Frecruencv 

1 

DMA  Headquarters 

10 

2 

DMA  Systems  Center 

8 

3 

DMA  Reston  Center 

3 

4 

DMA  Hydrographic/Topographic  Center 

16 

5 

DMA  Aerospace  Center 

9 

6 

Other  Telecommunication  Services  Center 

(1) 

Combat  Support  Center  (1) 

2 

7 

More  than  one  organization 

11 

8 

None 

3 

TOTAL 

62 

experience  other  than  their  present  position.  The  data  in 
tables. 12  and  13  suggest  that  the  DMA  experts  were 
functionally  mobile  and  received  much  of  their  experience  in 
two  or  three  DMA  organizational  components. 

Basic  Characteristics  of  the  LSMM  Model .  The  basic 
characteristics  of  the  LSMM  model  were  divided  into  three 
subject  areas.  These  are  (1)  the  basic  structure  of  the 
model  as  it  applies  to  the  logistics  support  of  automated 
cartographic  or  photogrammetric  systems;  (2)  the  design  or 
life  cycle  processes  of  the  model;  and  (3)  the  ILS  elements 
portrayed  in  the  model.  There  was  not  a  separate  subject 
area  for  the  principles  of  management  ("X"  axis  of  the  LSMM 
model)  as  it  was  defined  to  be  outside  the  scope  of  this 
study  (see  Scope  and  Limitations  section  of  Chapter  I)  . 
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The  responses  to  the  Likert  scale  questions  concerning 
the  basic  characteristics  of  the  LSMM  model  are  displayed  in 
Table  14.  Responses  to  multiple  choice  and  rank-ordered 
questions  concerning  the  basic  characteristics  of  the  LSMM 
model  are  provided  throughout  this  section.  Some  experts’ 
comments  are  highlighted  in  the  text  while  all  comments 
provided  in  round  one  of  the  Delphi  survey  are  found  in 
Appendix  F. 

Table  14 

Likert  Responses 
for  the 

Basic  Characteristics  of  the  LSMM  Model 
--  Round  One  Delphi  Survey 


Topic 

1 

Ratings 

2  3  4 

5 

Mean 

Consensus 

Basic 

Structure 

0 

0 

6 

23 

2 

3.87 

81% 

agree 

Design  or  Life  Cycle 

Basic 

Understanding 

0 

0 

0 

18 

13 

4.42 

100% 

agree 

Model 

Portrayal 

1 

5 

4 

19 

1 

3.47 

65% 

agree 

ILS 

Basic 

Understanding 

0 

0 

2 

18 

11 

4.29 

94% 

agree 

Model 

Portrayal 

3 

6 

6 

12 

3 

3.20 

Legend  for  Ratings 

5  =  highly  agree 
4  =  agree 

3  =  neither  agree  nor  disagree 
2  =  disagree 
1  =  highly  disagree 
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Basic  Structure.  Eighty-one  percent  of  the  DMA 
experts  agreed  that  the  LSMM  provided  the  basic  structure 
for  outlining  the  logistics  support  of  automated 
cartographic  or  photogrammetric  systems  (see  Table  14). 

The  experts  were  asked  how  they  would  change  the  LSMM.  One 
expert  felt  the  "Z"  axis  should  be  reversed  to  depict 
the  same  origin  as  the  "X"  and  "Y"  axes.  Another  expert 
thought  a  new  phase  entitled  "transition"  should  be  added  on 
the  "Y"  axis  between  production  and  operational  use. 

Design  or  Life  Cycle  Processes .  The  experts 
unanimously  agreed  that  an  understanding  of  design  or  life 
cycle  processes  was  important  for  logistics  support  planning 
of  automated  cartographic  or  photogrammetric  systems  (see 
Table  14).  Although  consensus  was  reached  for  the  question 
concerning  the  model  accurately  portraying  the  design  of 
life  cycle  phases  of  automated  cartographic  or 
photogrammetric  systems  employed  by  DMA,  the  percentage 
agreement  was  only  65%  (see  Table  14). 

The  experts  were  asked  to  rank  order,  from  one  to  six 
with  "1"  being  the  most  important,  the  design  or  life  cycle 
phases  in  terms  of  importance  for  logistics  support  planning 
of  DMA  automated  cartographic  or  photogrammetric  systems. 
Using  the  mean/rank  method,  the  mean  responses  from  round 
one  were  ranked  from  one  to  six.  The  only  consensus 
response  was  retirement,  which  was  ranked  sixth.  Table  15 
displays  the  six  design  or  life  cycle  phases,  the  rank  order 
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Table  15 


Design  or  Life  Cycle  Phases 
of  the  LSMM  Model 
--  Round  One  Delphi  Survey 


Rank  Order 


Phase 

X_ 

Frequencies 

2  3  4  5 

6 

Mean  or 
Consensus 

Detailed  Design 

9 

6 

13 

1 

2 

0 

2.39 

Advanced  Development 

7 

12 

1 

9 

2 

0 

2.58 

Production 

2 

5 

8 

14 

2 

0 

3.29 

Operational  Use 

5 

5 

6 

5 

10 

0 

3.32 

Conceptual 

8 

2 

3 

2 

12 

4 

3.65 

Retirement 

0 

1 

0 

0 

3 

27 

87%  agree 

frequencies,  and  the  round  one  mean  or  percentage  agreement 
for  each  phase. 

The  experts  were  questioned  on  how  they  would  change 
the  design  or  life  cycle  phases  proposed  in  the  LSMM.  Some 
experts  thought  the  design  or  life  cycle  phases  should  be 
changed  to  reflect  DMA  terminology  while  others  thought  the 
terminology  should  have  been  DOD  specific.  Another  expert 
stated 


. . .  the  operational  phase  is  conceived  as  merely  being  * 

the  use  of  the  "product".  In  reality,  life  cycle  phases 

1-4  normally  continue  to  reoccur  repeatedly  throughout 

the  operational  life  of  automated  cartographic  and 

photogrammetric  systems.  In  my  experience  seldom  if 

ever  has  a  production  system  been  the  same  "product"  at 

the  end  of  its  life  cycle  as  was  envisioned  at  the 

start  of  its  operational  use. 
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Integrated  Logistics  Support  ( ILS)  Elements .  The 


experts  reached  a  level  of  consensus  (94%  agreement)  that  an 
understanding  of  Integrated  Logistics  Support  processes  was 
important  for  logistics  support  planning  of  automated 
cartographic  or  photogrammetric  systems  (see  Table  14). 
Responses  were  mixed  (Round  one  mean  3.20)  on  whether  the 
model  accurately  portrayed  the  ILS  elements  applied  to 
automated  cartographic  or  photogrammetric  systems  employed 
by  DMA  (see  Table  14). 

For  the  ten  ILS  elements  portrayed  in  the  LSMM  model, 
the  experts  were  asked  to  rank  the  three  most  important  and 
the  three  least  important  ILS  elements  associated  with 
logistics  support  planning  of  automated  cartographic  or 
photogrammetric  systems. 

In  Table  16,  the  rank  order  frequencies  of  the  three 
most  important  ILS  elements  were  shown  using  "1"  to  denote 
the  first,  "2"  to  denote  the  second,  and  "3"  to  denote  the 
third  most  important  ILS  element.  A  weighting  factor  was 
assigned  to  each  response  to  determine  the  three  MOST 
IMPORTANT  ILS  elements  associated  with  logistics  support 
planning  of  automated  cartographic  or  photogrammetric 
systems.  A  value  of  five  was  assigned  to  each  ILS  element 
with  a  "1"  response;  a  value  of  three  was  assigned  to  each 
ILS  element  with  a  "2"  response;  and  a  value  of  one  was 
assigned  to  each  ILS  element  with  a  "3"  response.  The  total 
value  of  each  ILS  element  was  used  to  determine  the  overall 
ranking,  from  TOP  TO  BOTTOM,  of  the  MOST  IMPORTANT  ILS 
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Table  16 


Host  Important  ILS  Elements 

of  the  LSMM  Model  * 


--  Round  i 

One 

Delphi  Survey 

ILS  Elements 

1 

Rank  Order 
Frequencies 
2 

3 

Value  or 
Percent 
Consensus 

Maintenance  Planning 

19 

3 

4 

62%  agree 

Manpower/ Personnel 

4 

10 

4 

54 

Training 

3 

5 

6 

35 

Supply  Support 

1 

4 

6 

23 

Design  Interface 

2 

2 

2 

18 

Technical  Data 

2 

1 

3 

16 

Computer  Resources  Support 

0 

3 

1 

10 

Facilities 

0 

2 

3 

9 

Support  Equipment 

0 

1 

2 

5 

Packaging,  Handling, 
Storage  and  Transportation 

0 

0 

0 

0 

ELEMENTS  (see  Table  16).  The  only  consensus  response  was 
maintenance  planning,  which  was  found  to  be  the  most 
important  or  ranked  first. 

In  Table  17,  the  rank  order  frequencies  of  the  three 
least  important  ILS  elements  were  shown  using  "8"  to  denote 
the  eighth,  "9"  to  denote  the  ninth,  and  "10"  to  denote  the 
tenth  least  important  ILS  element.  A  weighting  factor  was 
assigned  to  each  response  to  determine  the  three  LEAST 
IMPORTANT  ILS  elements  associated  with  logistics  support 
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Table  17 


Least  Important  ILS  Elements 
of  the  LSMM  Model 
--  Round  One  Delphi  Survey 


ILS  Elements 

Rank  Order 
Frequencies 
10  9 

8 

Value  or 
Percent 
Consensus 

Packaging,  Handling, 

Storage  and  Transportation 

17 

7 

4 

110 

Design  Interface 

9 

4 

5 

62 

Facilities 

2 

6 

7 

35 

Computer  Resources  Support 

3 

2 

3 

24 

Support  Equipment 

0 

4 

3 

15 

Supply  Support 

0 

4 

3 

15 

Manpower/ Personnel 

0 

3 

0 

9 

Technical  Data 

0 

0 

2 

2 

Training 

0 

0 

0 

0 

Maintenance  Planning 

0 

0 

0 

0 

planning  of  automated  cartographic  or  photogrammetric 
systems.  A  value  of  five  was  assigned  to  each  ILS  element 
with  a  "10"  response;  a  value  of  three  was  assigned  to  each 
ILS  element  with  a  "9"  response;  and  a  value  of  one  was 
assigned  to  each  ILS  element  with  an  "8"  response.  The 
total  value  of  each  ILS  element  was  used  to  determine  the 
overall  ranking,  from  TOP  TO  BOTTOM,  of  the  LEAST  IMPORTANT 
ILS  ELEMENTS  (see  Table  17).  There  were  not  any  consensus 
responses  in  round  one  of  the  Delphi  survey. 
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The  experts  were  asked  if  any  of  the  ILS  elements  were 
not  applicable  to  automated  cartographic  or  photogrammetric 
systems  and  why  did  they  lack  applicability.  Some  experts 
believed  that  certain  elements  were  not  applicable. 

Specific  comments  were  directed  at  the  applicability  of  the 
design  interface  and  manpower/personnel  elements  found  in 
the  model .  One  expert  stated  "the  design  interface  is  often 
not  applicable  because  much  of  the  hardware  is  Commercial 
Off-the-shelf  (COTS)  and  we  have  little  input  into  its 
design".  Concerning  manpower  and  personnel,  an  expert 
stated 

. . .  manpower/personnel  may  not  be  applicable  in  the  DMA 
environment  because  of  our  structured  environment.  The 
time  frames  involved  in  justifying  and  staffing  new 
positions,  particularly  in  a  time  of  declining  budgets 
makes  it  difficult  to  get  the  people  needed  in  the  time 
frame  involved. 

Another  expert  presented  an  opposing  viewpoint  concerning 

the  applicability  of  the  ILS  elements.  The  expert  stated 

...  if  anyone  says  an  ILS  element  is  not  applicable  to 
automated  cartographic  or  photogrammetric  systems  I 
would  discount  his/her  entire  survey  -  the  individual 
obviously  doesn't  comprehend  ILS. 

Asked  if  they  would  change  any  of  the  ILS  elements  or 
add  any  elements  to  the  LSMM  model,  some  experts  responded 
that  security  is  a  major  issue  and  security  support  should 
b e  added  as  a  separate  ILS  element.  Another  expert  added 
that  ILS  usually  references  safety  as  part  of  the 
documentation  and  needs  to  be  added  to  the  model .  Critical 
to  the  model  an  expert  stated 
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the  model  fails  to  recognize  the  fundamental  nature 
of  these  systems,  which  is  that  they  are  a  part  of  a 
production  process .  These  processes  and  the  systems 
necessarily  change  and  evolve  as  improvements  are  made 
to  increase  productivity  and  as  new  or  changed  products 
are  introduced  into  the  production  line.  Automated 
cartographic  or  photogrammetric  systems  evolve  during 
their  life  cycle  so  that  they  are  often  very  different 
from  their  initial  configuration  and  use  in  operation. 
Changes  are  made  in  both  system  hardware  and  software 
to  effect  this  evolution.  The  ability  to  constantly 
modify  applications  software  is  particularly  crucial  to 
the  production  use  of  these  systems.  Applications 
software  is  very  much  an  extension  of  the  cartographer 
or  photogrammetrist .  It  is  probably  not  useful  or 
meaningful  to  try  to  separate  conceptually  software 
maintenance  from  software  development  or  enhancement. 
The  problem  with  the  model  is  that  it  is  linear,  when 
the  actual  life  cycle  for  these  systems  involves  an 
iterative  development  and  operational  cycle. 

The  final  question  presented  to  the  experts  in  the 

basic  characteristics  of  the  LSMM  model  portion  of  the 

survey  pertained  to  the  extent  to  which  DMA  employed  ILo  in 

logistics  support  requirements  planning  of  automated 

cartographic  or  photogrammetric  systems  (see  Table  18). 


Table  18 

Extent  of  ILS  Planning  for  DMA 
Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  One  Delphi  Survey 


Frequencies 

Topic  12345  Mean  Consensus 

Extent  of  ILS  Planning  2  7  16  4  0  2.76 

Legend  for  Frequencies 

1  =  No  extent  at  all 
3  =  To  a  moderate  extent 
5  =  To  a  very  great  extent 
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The  experts  responded  on  a  scale  from  one  to  five,  with  "l" 
representing  to  no  extent  at  all,  "3"  representing  to  a 
moderate  extent,  and  "5"  representing  to  a  very  great 
extent.  Table  18  presents  the  response  frequencies  and  the 
round  one  mean.  Consensus  was  not  reached  for  this  question 
in  round  one  of  the  survey. 

Unique  Factors  of  ILS  Elements .  In  the  third  section 
of  the  survey,  each  Delphi  expert  provided  comments  on  what 
factors,  within  each  of  the  j LS  elements  portrayed  in  the 
LSMM  model,  were  unique  and  required  logistics  support 
planning  for  automated  cartographic  or  photogrammetric 
systems.  For  each  ILS  element  a  list  of  factors  unique  to 
automated  cartographic  or  photogrammetric  systems  was 
collated  from  round  one  responses.  Table  19  provides  a 
sample  list  of  factors  unique  to  automated  cartographic  or 
photogrammetric  systems.  Complete  lists  of  factors  were 
arranged  by  ILS  element  and  displayed  in  Appendix  J  (Tables 
39  through  48). 

Life  Cycle  Determination  and  Design  Requirements .  The 
fourth  section  of  the  survey  required  experts  to  complete 
questions  concerning  when  in  the  life  cycle  of  automated 
cartographic  or  photogrammetric  systems  should  determination 
and  design  requirements  for  each  ILS  element  take  place. 

The  experts  responded  on  a  scale  from  one  to  five,  with  "1" 
representing  the  conceptual  phase .  "2"  representing  the 
advanced  development  phase,  "3"  representing  the  detailed 
design  and  development  phase,  "4"  representing  the 


Table  19 


Sample  List  of 

Supply  Support  Factors  Unique 
to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  One  Delphi  Survey 


1.  Adequate  shelf  stock  of  required  parts 

2.  COTS  equipment  has  impact  on  the  provisioning  process, 
proprietary  data  rights,  and  basic  supply  support 
concept 

3.  Dependent  on  local  purchase  due  to  unique  spares  not 
being  stocked  in  DOD  supply  system 

4.  Hardware  life  cycle  is  over  before  rystems  are  in  full 
production 

5.  Initial  spare  start-up  quantities 

6.  Large  computer  maintenance  contracts 

7.  Optical  components 

8.  Staging  of  spares 

9.  Steady  state  life  cycle  spare  requirements 

10.  Tapes,  removable  drives,  plotter  pens,  plotter  paper 

11.  Unique  systems  need  unique  spares,  requiring  long  lead 
time 


production  phase,  and  "5"  representing  the  operational  use 
phase.  Table  20  lists  the  ILS  element,  the  life  cycle 
response  frequencies,  and  the  round  one  mean  or  consensus 
percent  agreement. 

The  experts  could  not  reach  consensus  on  when  in  the 
life  cycle  determination  and  design  requirements  for  the 
maintenance  planning,  manpower/personnel,  computer  resources 
support,  facilities,  and  design  interface  elements  should 
take  place.  The  round  one  means  for  these  elements  were 
displayed  in  Table  20.  The  experts  did  reach  consensus  on 
when  in  the  life  cycle  determination  and  design  requirements 
for  the  supply  support,  support  equipment,  technical  data, 
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Table  20 


Life  Cycle  Determination  and  Design  Requirements 

for  ILS  Elements 
--  Round  One  Delphi  Survey 


ILS  Elements 

Life  Cycle  Response 
Frequencies 

1  2  3  4  5 

Mean  or 
Consensus 

Maintenance  Planning 

4 

16 

6 

2 

0 

2.21 

Manpower /Personnel 

3 

8 

17 

l 

0 

2.55 

Supply  Support 

0 

5 

21 

3 

0 

68%  agree 

Support  Equipment 

0 

7 

21 

2 

0 

68%  agree 

Technical  Data 

3 

3 

19 

3 

0 

62%  agree 

Training 

0 

3 

23 

5 

0 

74%  agree 

Computer  Resources  Support 

3 

13 

13 

1 

0 

2.40 

Facilities 

3 

12 

15 

0 

0 

2.40 

Packaging,  Handling, 

Storage  and  Transportation 

1 

0 

22 

6 

0 

71%  agree 

Design  Interface 

5 

12 

8 

2 

0 

2.26 

Legend  for  Life  Cycle  Response  Frequencies 

1  =  Conceptual  phase 

2  =  Advanced  development  phase 

3  =  Detailed  Design  and  Development  phase 

4  =  Production  phase 

5  =  Operational  use  phase 


training,  and  packaging,  handling,  storage,  and 
transportation  elements  should  take  place.  The  experts 
agreed  that  each  of  these  elements  required  determination 
and  design  during  the  detailed  design  and  development  phase 
(see  Table  20) . 
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The  last  question  presented  to  the  experts  in  round  one 
of  the  Delphi  survey  was  open-ended  and  pertained  to  how 
well  DMA  applies  ILS  or  any  other  logistics  support  planning 
process  to  automated  cartographic  or  photogrammetric 
systems.  Most  comments  provided  by  the  experts  were  similar 
to  the  following: 

...  the  one  area  that  DMA  historically  has  ignored  is 
the  application  of  ILS  during  the  conceptual  phase  of 
development  rather  than  at  the,  too  typically, 
unfortunately,  operational  phase.  MARK  85  and  MARK  90 
systems  are  examples  of  how  DMA  is  changing  with 
respect  to  ILS.  DMA  needs  to  follow  the  formal 
processes  of  the  life  cycles  of  systems  in  a  more 
rigorous  manner  for  all  systems,  as  is  being  done  for 
MARK  90. 

Another  expert  stated 

...  we  take  too  long  to  start  the  process,  as  a  result 
ILS  doesn't  do  what  it  is  intended  to  do  (decisions 
have  already  been  made). 

Round  Two  Results 

The  response  rate  for  round  two  of  the  Delphi  survey 
was  81%  (25  of  31)  for  those  participants  from  the  first 
round  and  71%  (25  of  35)  for  the  original  survey  respondent 
group.  Only  round  one  Delphi  respondents  were  mailed  a 
round  two  survey  questionnaire.  The  numerical  responses  and 
open-ended  comments  provided  by  the  experts  in  round  tv;o 
continued  to  be  of  great  value  in  the  research  effort. 

Feedback  from  round  one  results  of  the  survey  was 
provided  to  the  experts  in  round  two.  Delphi  respondents 
were  asked  to  complete  questions  in  the  following  three 
areas:  (1)  basic  characteristics  of  the  LSMM  model; 
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(2)  unique  factors  of  ILS  elements  associated  with  automated 
cartographic  or  photogrammetric  systems;  and  (3)  life  cycle 
determination  and  design  requirements  of  the  ILS  elements. 
The  experts'  responses  to  these  areas  are  reviewed  below 

Basic  Characteristics  of  the  LSMM  Model .  The  experts 
again  were  asked  to  provide  numerical  responses  and  comments 
on  the  three  subject  areas  which  pertained  to  the  basic 
characteristics  of  the  LSMM  model.  These  were  (1)  the  basic 
structure  of  the  model  as  it  applies  to  the  logistics 
support  of  automated  cartographic  cr  photogrammetric 
systems;  (2)  the  design  or  life  cycle  processes  of  the 
model;  and  (3)  the  ILS  elements  portrayed  in  the  model. 

The  responses  to  the  Likert  scale,  multiple  choice  and 
rank-ordered  questions  concerning  the  basic  characteristics 
of  the- LSMM  model  are  displayed  throughout  this  section. 
Select  round  two  experts'  comments  are  highlighted  in  the 
text  and  all  comments  provided  in  round  two  of  the  Delphi 
survey  are  found  in  Appendix  H. 

* 

Basic  Structure .  After  reviewing  round  one 

comments,  the  DeJphi  experts  were  asked  again  to  provide 

comments  on  how  they  would  change  the  LSMM.  One  expert 

refuted  some  of  the  round  one  comments  and  stated 

...  disagree  with  adding  resources  =  $$.  Resources  are 
a  limiting  factor  to  the  individual  categories  as  well 
as  to  the  model  as  a  whole.  "Transition  is  not  a 
separate  logistics  step;  one  could  put  a  "transition" 
step  between  every  life  cycle  phase.  Test  equipment 
should  not  be  a  separate  element  because  it  is  a  type 
of  "support  equipment".  DMA  should  use  DOD  terminology 
rather  than  its  own. 
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Another  expe  t  supported  the  consensus  agreement  that  the 
LSMM  provides  a  basic  structure  for  outlining  the  logistics 
support  of  automated  cartographic  or  photogrammetric 
systems.  The  expert  stated 

...  there  is  nothing  wrong  with  the  LSMM  model.  It  is 
only  how  the  LSMM  model  is  implemented  -  that  is  the 
problem. 

Design  or  Life  Cycle  Processes .  The  experts  were 
asked  again  to  rank  the  five  design  or  life  cycle  phases  in 
terms  of  importance  for  logistics  support  planning  of  DMA 
automated  cartographic  or  photogrammetric  systems.  Round 
one  results  showed  the  only  consensus  response  was 
retirement,  which  was  ranked  sixth.  For  the  remaining  five 
design  or  life  cycle  phases,  the  experts  were  provided  round 
one  response  means  as  feedback.  Table  21  displays  the  six 
design  or  life  cycle  phases,  the  round  two  rank  order 
frequencies,  and  the  round  two  mean  or  percentage  agreement 
for  each  phase. 

Consensus  agreement  was  reached  for  the  conceptual 
phase  in  round  two  of  the  survey  (see  Table  21).  The 
conceptual  phase  was  ranked  fifth  in  terms  of  importance  for 
logistics  support  planning  of  DMA  automated  cartographic  or 
photogrammetric  systems.  Although  consensus  was  not  reached 
for  the  remaining  life  cycle  phases,  the  round  one  and  two 
means  showed  detailed  design  and  development  was  ranked 
first,  advanced  development  second,  production  third,  and 
operational  use  fourth  (see  Tables  15  and  21). 
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Table  21 


Design  or  Life  Cycle  Phases 
of  the  LSMM  Model 
--  Round  Two  Delphi  Survey 


Phase 

Detailed  Design 
Advanced  Development 
Production 
Operational  Use 
Conceptual 
Retirement 


Rank  Order 

Frequencies  Mean  or 

123456  Consensus 


11  8  4 

5  10  5 

2  3  6 

14  8 


2  0  0 

3  2  0 

12  2  0 

6  6  0 

2  15  0 


1.88 

2.48 
3.36 

3.48 

60%  agree 
87%  agree 


6  0  2 
Round  one  consensus: 


Experts  were  asked  again  to  provide  comments  on  how 
they  would  change  the  design  or  life  cycle  phases  proposed 
in  the  LSMM.  Their  comments  supported  the  agreement  reached 
in  round  one  that  the  model  portrayed  the  design  or  life 
cycle  phases  of  DMA  automated  cartographic  or 
photogrammetric  systems.  Other  comments  focused  on  the 
design  or  life  cycle  terminology  used  in  the  LSMM. 

One  expert  suggested  that  the  design  or  life  cycle 
phase  terminology  used  in  the  LSMM  model  be  changed  to  DMA 
terminology.  Table  22  presented  the  equivalent  comparison 
of  life  cycle  terminology  suggested  by  the  expert.  Other 
experts  agreed  the  life  cycle  terminology  should  adhere  to 
DOD  life  cvcle  phase  terminology. 
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Table  22 


DMA  Terminology 
for 

Life  Cycle  Phases  Portrayed  in  LSMM  Model 
--  Round  Two  Delphi  Survey 


LSMM  Terminology 
Conceptual 

Advanced  Development 

Detailed  Design 

Production 

(No  Counterpart) 

Operational  Use 

Retirement 


DMA  Terminology 

Design  Concept 
Preliminary  Design 
Critical  Design 
Build 

Transition 

Production  Operations 
Excessing 


Integrated  Logistics  Support  ( ILS)  Elements .  In 
round  one  experts  agreed  (94%)  that  understanding  ILS  was 
important  for  logistics  support  planning  of  automated 
cartographic  or  photogrammetric  systems.  However,  the 
experts  could  not  reach  a  level  of  consensus  on  whether  the 
model  accurately  portrayed  the  ILS  elements  applied  to 
automated  cartographic  or  photogrammetric  systems  employed 
by  DMA.  Results  from  round  two  of  the  survey  indicated  that 
68%  agreed/highly  agreed  that  the  model  accurately  portrayed 
the  ILS  elements  applied  to  automated  cartographic  or 
photogrammetric  systems  employed  by  DMA  (see  Table  23). 

For  the  ten  ILS  elements  portrayed  in  the  LSMM  model, 
the  experts  were  once  again  asked  to  rank  most  important  and 
least  important  ILS  elements  associated  with  logistics 
support  planning  of  automated  cartographic  or 
photogrammetric  systems. 
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Table  23 


Likert  Responses 
for  the 

Basic  Characteristics  of  the  LSMM  Model 
--  Round  Two  Delphi  Survey 


Ratings 

Tonic  12345  Mean  Consensus 

ILS 


Model  Portrayal  0  3  5  13  4  3.72  68%  agree 

Legend  for  Ratings 

5  =  highly  agree 
4  =  agree 

3  =  neither  agree  nor  disagree 
2  =  disagree 
1  =  highly  disagree 


For  round  two  of  the  survey,  the  experts  were  only 
required  to  rank  the  second  and  third  most  important  ILS 
elements.  Results  from  round  one  of  the  survey  indicated 
that  maintenance  planning  was  ranked  first  with  a  62% 
consensus  agreement.  In  Table  24,  the  rank  order 
frequencies  of  the  second  and  third  most  important  ILS 
elements  were  shown  using  ”2"  to  denote  the  second,  and  "3" 
to  denote  the  third  most  important  ILS  element.  A  weighting 
factor  was  assigned  to  each  response  to  determine  the  second 
and  third  MOST  IMPORTANT  ILS  elements  associated  with 
logistics  support  of  automated  cartographic  or 
photogrammetric  systems. 
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Table  24 


Most  Important  ILS  Elements 
of  the  LSMM  Model 
--  Round  Two  Delphi  Survey 


Rank  Order 
Frequencies 


ILS  Elements  2 _ 3 

Maintenance  Planning  Round  One  Consensus 

Manpower/Personnel  14  5 

Design  Interface  5  2 

Training,  2  8 

Supply  Support  3  4 

Facilities  1  1 

Computer  Resources  Support  0  3 

Technical  Data  0  2 

Support  Equipment  0  0 

Packaging,  Handling, 

Storage  and  Transportation  0  0 


Value  or 
Percent 
Consensus 

62%  agree 

47 

17 

14 

13 

4 

3 

2 

0 

0 


A  value  of  three  was  assigned  to  each  ILS  element  with 
a  "2"  response;  and  a  value  of  one  was  assigned  to  each  ILS 
element  with  a  "3"  response.  The  total  value  of  each  ILS 
element  determined  the  overall  ranking,  from  TOP  TO  BOTTOM, 
of  the  MOST  IMPORTANT  ILS  ELEMENTS  (see  Table  24).  Although 
consensus  was  not  reached  for  any  of  the  elements  in  round 
two,  a  weighted  response  value  of  47  strongly  supports 
ranking  manpower/personnel  second.  Design  interface  ranked 
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third  with  a  weighted  response  value  of  17  slightly  ahead  of 
training  and  supply  support. 


In  Table  25,  the  rank  order  frequencies  of  the  three 
least  important  ILS  elements  were  shown  using  "8"  to  denote 
the  eighth,  "9”  to  denote  the  ninth,  and  "10”  to  denote  the 
tenth  least  important  ILS  element.  A  weighting  factor  was 

Table  25 

Least  Important  ILS  Elements 
of  the  LSMM  Model 
--  Round  Two  Delphi  Survey 


ILS  Elements 

Rank  Order 
Frequencies 
10  9 

8 

Value  or 
Percent 
Consensus 

Packaging,  Handling, 

Storage  and  Transportation 

23 

0 

1 

92%  agree 

Design- Interface 

1 

11 

4 

42 

Facilities 

1 

3 

8 

22 

Support  Equipment 

0 

3 

7 

16 

Computer  Resources  Support 

0 

3 

0 

9 

Technical  Data 

0 

2 

1 

7 

Manpower /Personnel 

0 

2 

1 

7 

Supply  Support 

0 

0 

3 

3 

Training 

0 

1 

0 

3 

Maintenance  Planning 

0 

0 

0 

0 
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assigned  to  each  response  to  determine  the  LEAST  IMPORTANT 
ILS  elements  associated  with  logistics  support  planning  of 
automated  cartographic  or  photogrammetric  systems. 

A  value  of  five  was  assigned  to  each  ILS  element  with  a 
*  "10"  response;  a  value  of  three  was  assigned  to  each  ILS 

element  with  a  "9"  response;  and  a  value  of  one  was 
assigned  to  each  ILS  element  with  an  "8"  response.  The 
total  value  of  each  ILS  element  determined  the  overall 
ranking,  from  TOP  TO  BOTTOM,  of  the  LEAST  IMPORTANT  ILS 
ELEMENTS  (see  Table  25).  The  only  consensus  response  was 
packaging,  handling,  storage,  and  transportation,  which  was 
found  to  be  the  least  important  or  ranked  tenth.  The 
experts  ranked  design  interface  ninth  with  a  weighted  value 
of  42  and  facilities  eighth  with  a  weighted  value  of  22. 

The  experts  were  asked  again  if  any  of  the  ILS  elements 
were  not  applicable  to  automated  cartographic  or 
photogrammetric  systems  and  why  they  were  not  applicable. 

The  experts'  comments  supported  the  applicability  of  ILS  to 
automated  cartographic  or  photogrammetric  systems.  One 
expert  stated 

...  none  -  The  least  important  call  is  tough,  even 
packaging,  handling,  storage  and  transportation  is 
important  with  a  planning  factor  of  1000  failed  LRU's 
v  a  month,  most  of  which  must  be  shipped  out  for  depot 

maintenance. 

Another  expert  supported  the  same  idea.  The  expert  stated 

All  parts  of  ILS  are  important,  but  unless  we  plan  to 
move  systems,  packaging,  handling,  storage  and 
transportation  is  probably  the  least  important.  I 
think  some  participants  missed  the  point  concerning 
manpower  and  personnel  -  in  ILS  this  usually  refers  to 
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the  people  maintaining  the  system,  not  the  operators. 
Remember,  this  is  a  logistics  model.  Thrre  is  another 
whole  world  that  analyzes  what  the  system  does,  how 
does  it  work,  and  who  operates  it. 

Asked  again  if  they  would  change  any  of  the  ILS 

elements  or  add  any  elements  to  the  LSMM  model,  experts 

again  responded  that  security  is  a  major  issue  and  security 

support  should  be  added  as  a  separate  ILS  element  (see 

Appendix  H).  One  expert  presented  a  broader  view  of 

changing  the  ILS  structure.  The  expert  stated 

...  assigning  ILS  tasks  to  a  contractor  is  possible, 
but  that  doesn't  eliminate  those  requirements  or 
abrogate  our  responsibility  for  ensuring  ILS 
requirements  are  actually  met. 

The  question  concerning  the  extent  to  which  DMA 
employed  ILS  in  logistics  support  planning  of  automated 
cartographic  or  photogrammetric  systems  was  asked  again  in 
round  two.  The  experts  responded  on  a  scale  from  one  to 
five,  with  "1"  representing  to  no  extent  at  all,  "3" 
representing  to  a  moderate  extent,  and  "5"  representing  to  a 
very  great  extent.  Table  26  presents  the  response 
frequencies  and  the  percent  consensus  agreement.  Seventy- 
six  percent  of  the  experts  agreed  that  DMA  does  employ,  to  a 
moderate  extent,  ILS  in  logistics  support  requirements 
planning  of  automated  cartographic  or  photogrammetric 
systems  (see  Table  26). 

Unique  Factors  of  ILS  El ements .  In  round  one  of  the 
Delphi  survey  a  list  of  factors  unique  to  automated 
cartographic  or  photogrammetric  systems  was  collated  for 
each  ILS  element.  During  round  two,  the  experts  were  asked 
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Table  26 


Extent  of  ILS  Planning 
for  DMA 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  Two  Delphi  Survey 


Frequencies 

Topic  12345  Mean  Consensus 

Extent  of  ILS  Manning  0  5  19  1  0  2.84  76%  agree 

Legend  for  Frequencies 

1  =  No  extent  at  all 
3  =  To  a  moderate  extent 
5  =  To  a  very  gioat  extent 


to  review  each  list  and  provide  the  three  most  important 
factors,  for  each  ILS  alement,  that  were  unique  to  automated 
cartographic  or  photogrammetric  systems.  Table  27  provides 
a  sample  ranking  of  factors  unique  to  automated  cartographic 
or  photogrammetric  systems.  Appendix  K,  Tables  49  through 
58,  lists  the  ranking  and  frequency  count  of  each  factor  for 

the  ten  ILS  elements  in  the  LSMM  model. 

* 

Life  Cycle  D ^termination  and  Design  Requirements . 

During  round  two,  experts  again  were  required  to  complete 
questions  concerning  when  in  the  life  cycle  of  automated 
cartographic  or  photogrammetric  systems  should  determination 
and  design  requirements  for  each  ILS  element  take  place. 

In  round  one,  the  experts  could  not  reach  consensus  on  when 
in  the  life  cycle  determination  and  design  requirements  for 
the  maintenance  planning,  manpower/personnel,  computer 
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Table  27 


Sample  Ranking  of 
Supply  Support  Factors  Unique 
to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  Two  Delphi  Survey 

_ * 

Frequency 

1.  Adequate  shelf  stock  of  required  parts  18 

2.  Unique  systems  need  unique  spares,  requiring  13 

long  lead  time 

3.  COTS  equipment  has  impact  on  the  provisioning  10 

process,  proprietary  data  rights,  and  basic 

supply  support  concept 

4.  Initial  spare  start-up  quantities  8 

5.  Hardware  life  cycle  is  over  before  systems  are  7 

in  full  production 

6.  Dependent  on  local  purchase  due  to  unique  5 

spares  not  being  stocked  in  DOD  supply  system 

7.  Staging  of  spares  5 

8.  Large  computer  maintenance  contracts  4 

9.  Steady  state  life  cycle  spare  requirements  3 

10.  Optical  components  1 

11.  Tapes,  removablt  drives,  plotter  pens,  1 

plotter  paper 


resources  support,  facilities,  and  design  interface  elements 
should  take  place.  For  these  five  elements  the  experts 
again  responded  on  a  scale  from  one  to  five,  with  "1" 
representing  the  conceptual  phase,  "2"  the  advanced 
development  phase,  "3"  the  detailed  design  and  development 
phase,  "4"  the  production  phase,  and  *'5"  the  operational  use 
phase . 

d 

Table  28  lists  the  ILS  elements,  the  life  cycle 
response  frequencies,  and  the  round  two  mean  or  consensus 

* 

percent  agreement. 


90 


Table  28 


Life  Cycle  Determination  and  Design  Requirements 

for  ILS  Elements 
--  Round  Two  Delphi  Survey 


Life 

>  cy 

cle 

Response 

ILS  Elements 

1 

Frequencies 

2  3  4 

5 

Mean  or 
Consensus 

Maintenance  Planning 

2 

15 

7 

1 

0 

60%  agree 

Manpower/ Personnel 

0 

8 

16 

1 

0 

64%  agree 

Computer  Resources  Support 

y 

J. 

12 

11 

* 

A 

0 

2.48 

Faci 1 ities 

0 

10 

14 

1 

0 

2.64 

Design  Interface 

1 

15 

9 

0 

0 

60%  agree 

Legend  for  Life  Cycle  Response  Frequencies 

1  =  Conceptual  phase 

2  =  Advanced  development  phase 

3  =  Detailed  Design  and  Development  phase 

4  =  Production  phase 

5"  Operational  use  phase 


During  round  two,  the  experts  did  reach  consensus  on 
when  in  the  life  cycle  determination  and  design  requirements 
for  the  maintenance  planning,  manpower/personnel,  and  design 
interface  elements  should  take  place.  The  experts  agreed 
maintenance  planning  ana  design  interface  required 
determination  and  design  during  the  advanced  development 
phase;  and  manpower/personnel  required  determination  and 
design  during  the  detailed  design  and  development  phase  (see 
Table  28) . 


91 


As  in  round  one  of  the  survey,  the  last  question 
presented  to  the  experts  was  open-ended  and  pertained  to  how 
well  DMA  applies  ILS  or  any  other  logistics  support  planning 
process  to  automated  cartographic  or  photogrammetric 

systems.  One  expert  stated  * 

...  interesting  perspectives,  many  think  earlier  when 
in  reality  all  you  get  is  less  firmness.  Best  time  to 
finalize  an  ILS  plan  is  when  building  something. 

Another  expert  presented  an  opposing  viewpoint  on  the 

application  of  ILS  for  DMA  automated  cartographic  or 

photogrammetric  systems.  The  expert  stated 

...  in  general,  life  cycle  costs  are  not  addressed  well 
in  DMA.  The  objective  of  system  developers  is  often  to 
minimize  development  cost  at  the  expense  of  total 
system  life  cycle  costs.  Because  of  the  nature  of  our 
mission  and  the  evolution  of  our  systems  over  what  is 
typically  a  very  long  life  cycle,  flexibility, 
adaptability,  and  long  term  maintainability  should  be 
given  more  consideration  in  the  design  process  than 
have  frequently  been  the  case. 

Many  of  the  comments  expressed  the  same  idea  concerning  ILS 

and  DMA  automated  systems.  One  expert  summed  up  these 

comments  and  stated  "we  simply  haven't  done  a  good  job  here 

-  we  constantly  play  catch  up". 


Conclusion 

This  chapter  described  the  results  from  two  iterative 
rounds  of  Delphi  questioning.  The  Delphi  technique  polled 
DMA  experts  in  an  attempt  to  reach  consensus  on  the 
attributes  and  characteristics  of  a  descriptive  model 
required  for  logistics  support  planning  of  automated 
cartographic  or  photogrammetric  systems.  Feedback  from  31 
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DMA  experts  in  round  one  of  the  survey  and  25  in  round  two 
provided  data  required  to  answer  the  three  research 
questions  proposed  in  Chapter  I.  Chapter  V  summarizes  the 
findings  to  the  research  questions  and  develops  a 
descriptive  model  to  address  the  logistical  support 
requirements  for  automated  cartographic  or  photogrammetric 
systems.  Chapter  VI  presents  contributions  of  this  research 
and  recommendations  for  future  research. 
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Anal vsis  of  Resul ts 

The  Delphi  method  provided  the  means  for  constructing  a 
descriptive  model  by  polling  expert  DMA  administrative, 
research  and  development,  support  and  maintenance,  and 
production  personnel  and  reaching  a  level  of  consensus  on 
logistical  support  planning  requirements  applicable  to 
automated  cartographic  or  photogrammetric  systems. 

The  validity  of  the  descriptive  model  was  ensured  by 
the  participation  of  DMA  experts  using  the  iterative  Delphi 
process.  Demographic  results  supported  the  level  of 
expertise  possessed  by  DMA  survey  participants.  Seventy- 
one  percent  of  the  participants  had  eight  or  more  years 
experience  with  automated  cartographic  or  photogrammetric 
systems.  The  experts  retained  a  broad  base  of  expertise  as 
much  of  their  experience  was  received  in  two  or  three  DMA 
organizational  components.  The  experts  also  demonstrated  a 
high  level  cf  educational  expertise.  Seventy-four  percent 
of  the  participants  had  a  graduate  degree  or  higher  level  of 
education. 

Results  from  the  Delphi  survey  concluded  Ostrofsky's 
Logistics  Systems  Management  Matrix  ( LSMM)  provided  the 
basic  structure  for  determining  logistics  support  procedures 
for  automated  cartographic  or  photogrammetric  systems 
employed  by  DMA. 
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The  results  collected  from  two  iterative  rounds  of 
Delphi  questioning  provided  the  necessary  data  to  answer  the 
three  research  questions  proposed  in  Chapter  I.  The 
research  questions  will  be  addressed  below. 

Research  Question  One: 

Can  DMA  experts  (administrative,  research  and 
development,  production,  operations  support  and 
maintenance)  reach  a  level  of  consensus  on  the 
characteristics  and  attributes  of  a  logistics  support 
model  applicable  to  automated  cartographic  or 
photogrammetric  systems? 

The  DMA  Delphi  experts  verified  the  LSMM  as  a 
descriptive  model  portraying  the  necessary  logistics 
characteristics  and  attributes  required  to  support  automated 
cartographic  or  photogrammetric  systems.  As  defined  in 
Chapter  I,  the  principles  of  management  dimension  portrayed 
in  the- LSMM  was  outside  the  scope  of  this  research  effort. 

A  division  of  the  survey,  entitled  the  basic  characteristics 
of  the  LSMM  model,  included  three  categories  required  to 
answer  research  question  one.  These  sections  consisted  of: 
(1)  the  basic  structure  of  the  LSMM;  (2)  the  design  or  life 
cycle  processes  of  the  model;  and  (3)  the  ILS  elements 
portrayed  in  the  model. 

The  first  round  of  the  survey  provided  consensus  on  the 
basic  structure  of  the  LSMM.  Eighty-one  percent  of  the 
experts  agreed  the  descriptive  model  illustrated  the 
necessary  requirements  for  logistics  support  planning  of 
automated  cartographic  or  photogrammetric  systems.  This 
finding  agreed  with  those  presented  in  the  literature  review 
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(Chapter  II).  The  research  indicated  the  LSMM  model  forms 
the  basis  for  proper  implementation  of  a  logistics  support 
plan  for  any  given  system  (Ostrofsky,  1986:8).  Open-ended 
responses  from  this  section  (see  Appendices  F  and  H) 
primarily  referenced  terminology  used  in  the  model  and  not 
the  actual  structure  of  the  model . 

Although  the  experts  unanimously  agreed  on  the 
requirement  for  understanding  design  or  life  cycle  phases, 
they  barely  reached  consensus  on  whether  the  descriptive 
model  accurately  portrayed  the  design  or  life  cycle  phases 
of  automated  cartographic  or  photogrammetric  systems 
employed  by  DMA.  Based  on  the  open-ended  responses,  the 
Delphi  experts  determined  the  design  or  life  cycle 
terminology  should  have  reflected  specific  DMA  or  DOD 
terminology  (see  Appendices  F  and  H).  The  use  of 
Finkelstein's  design  or  life  cycle  terminology  provided  a 
common  basis  of  understanding  for  all  experts.  The  design 
or  life  cycle  phases  illustrated  in  the  descriptive  model 
accurately  portrayed  the  entire  life  cycle  of  a  system; 
therefore,  they  were  found  to  be  acceptable  (Ostrofsky, 
1986:7) . 

The  experts  also  ranked  the  six  design  or  life  cycle 
phases  in  terms  of  importance  for  logistics  support  planning 
of  DMA  automated  cartographic  or  photogrammetric  systems. 
Consensus  was  reached  on  two  of  the  design  or  life  cycle 
phases.  The  conceptual  phase  ranked  fifth  and  the 
retirement  phase  sixth.  Logistics  support  planning  of 
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automated  cartographic  or  photogrammetric  systems  must  take 
place  early  in  the  system  life  cycle  (Materna  and  Andrews, 
1988:5-4).  Although  the  experts  ranked  the  conceptual  phase 
fifth,  the  literature  supported  the  remaining  rankings. 

Round  one  and  round  two  results  revealed  identical  findings. 
The  experts  ranked  detailed  design  and  development  first, 
advanced  development  second,  production  third,  and 
operational  use  fourth. 

As  with  the  design  or  life  cycle  phases,  the  experts 
highly  agreed  that  an  understanding  of  Integrated  Logistics 
Support  (ILS)  was  required  for  logistics  support  planning  of 
automated  cartographic  or  photogrammetric  systems.  Although 
the  experts  reached  consensus,  only  68%  agreed  the  model 
accurately  portrayed  the  ILS  elements  applied  to  automated 
cartographic  or  photogrammetric  systems  employed  by  DMA. 
Open-ended  responses  strongly  recommended  security  support 
be  added  to  the  ILS  dimension  of  the  descriptive  model . 
Security  considerations  are  a  major  factor  in  the  design  of 
all  DMA  systems;  therefore,  this  element  was  added  to  the 
descriptive  model . 

The  experts  ranked  the  ten  ILS  elements  in  terms  of  the 
three  most  important  and  the  three  least  important  for 
logistics  support  planning  of  automated  cartographic  or 
photogrammetric  systfems. 

The  experts  reached  consensus  that  maintenance  planning 
ranked  first.  This  was  consistent  with  the  findings 
presented  in  the  literature  review.  The  literature  stated 
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maintenance  planning  is  the  first  of  the  ten  ILS  elements  a 
logistics  manager  must  evaluate  (ILS  Guide,  1986:7-5). 
Although  consensus  was  not  reached  for  the  remaining  ILS 
elements  in  degree  of  greatest  importance,  the  manpower  and 
personnel  el«ment  ranked  second  with  a  high  weighted  value. 
Responses  from  open-ended  questions  expressed  a  need  to 
retain  competent  support  personnel  required  to  maintain 
automated  cartographic  or  photogrammetric  systems.  Design 
interface,  training,  and  supply  support  all  ranked  close  to 
the  third  position  with  only  four  weighted  value  points 
separating  the  three  elements. 

In  terms  of  least  importance,  the  experts  reached 
consensus  and  ranked  packaging,  handling,  storage,  and 
transportation  t-enth.  Past  experience  supported  this 
finding  as  DMA,  with  contractor  support,  delivered 
various  automated  systems  with  relatively  few  problems. 
Although  consensus  was  not  reached  for  the  remaining  ILS 
elements  in  terms  of  least  importance,  the  design  interface 
element  ranked  ninth  with  a  high  weighted  value.  Experts 
exhibited  a  poor  understanding  of  the  design  interface 
element  in  their  open-ended  responses.  This  explained  the 
inconsistent  third  highest  and  ninth  lowest  rankings.  With 
new  facilities  already  constructed  for  MARK  90  systems,  the 
experts  ranked  the  facilities  element  eighth. 

The  experts  provided  responses  to  a  question  concerning 
the  extent  to  which  DMA  employed  ILS  in  logistics  support 
planning  of  automated  cartographic  or  photogrammetric 
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systems.  The  experts  reached  consensus  that  DMA  employs,  to 
a  moderate  extent,  ILS  in  logistics  support  requirements 
planning  of  automated  cartographic  or  photogrammetric 
systems.  The  creation  of  a  Directorate  for  Acquisition, 
Installation,  and  Logistics  substantiated  DMA’s  commitment 
to  ILS  for  automated  cartographic  or  photogrammetric 
systems.  The  literature  encouraged  the  direction  of  DMA 
towards  an  ILS  concept.  Although  automated  cartographic  or 
photogrammetric  systems  do  not  attract  the  attention 
warranted  by  large  weapon  systems,  the  evaluation  of  their 
ILS  elements  remains  alike  (Materna  and  Andrews,  1988:5-4). 

Figure  15  illustrates  the  descriptive  model  developed 
from  a  review  of  the  literature  and  analysis  of  the  Delphi 
survey  results.  The  model  portrayed  in  Figure  15  was 
identical  to  the  LSMM  model  in  Figure  2  with  one  exception. 
The  security  element  added  to  the  ILS  axis  supported  the 
feedback  from  experts  requiring  inclusion  of  this  element  in 
the  descriptive  model .  Consensus  agreement  reached  on  the 
characteristics  and  attributes  of  the  descriptive  model 
validated  the  model  in  terms  of  applicability  to  DMA 
automated  cartographic  or  photogrammetric  systems. 

Research  Question  Two: 

What  factors  within  the  ILS  elements  are  unique  and 
applicable  to  automated  cartographic  or  photogrammetric 
systems? 

Round  one  of  the  survey  provided  expert  feedback  on 
what  factors,  within  each  of  the  ILS  elements  portrayed  in 
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Figure  15.  Descriptive  Model  of  Logistics  Support 

Requirements  for  DMA  Automated  Cartographic 
or  Photogrammetric  Systems 
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the  LSMM  model,  were  unique  to  logistics  support  planning 
for  automated  cartographic  or  photogrammetric  systems.  A 
list  of  factors  unique  to  automated  cartographic  or 
photogrammetric  systems  was  collated  from  round  one 
responses  (see  Appendix  J).  Round  two  of  the  survey 
required  experts  to  review  the  list  and  provide  the  three 
most  important  factors,  for  each  ILS  element,  that  he/she 
determined  were  unique  to  automated  cartographic  or 
photogrammetric  systems  (see  Appendix  K).  Some  experts 
commented  that  some  of  the  factors  identified  were  not 
solely  unique  to  automated  cartographic  or  photogrammetric 
systems.  Nevertheless,  these  factors  were  analyzed  as  they 
applied  to  logistics  support  planning  for  automated 
cartographic  or  photogrammetric  systems.  The  following 
analysis  examines  the  experts'  three  most  important 
responses  for  each  ILS. 

Maintenance  Planning.  Table  29  illustrates  the  three 
most  important  maintenance  planning  factors  unique  to 
automated  cartographic  or  photogrammetric  systems. 

The  determination  of  whether  the  maintenance  support 
will  be  in-house  or  contract  allows  managers  to  plan  for  the 
development  of  all  other  logistics  support  elements  (ILS 
Guide,  1986:7-5).  Planning  requirements  for  manpower  and 
personnel  staffing  levels,  supply  support  inventory  levels, 
and  procurement  of  support  equipment  are  determined  based  on 
whether  the  maintenance  will  be  in-house  or  contract. 


Table  29 


Three  Most  Important  Maintenance  Planning  Factors 

Unique  to 

Automated  Cartographic  or  Photogrammetric  Systems 


1.  Determine  whether  maintenance  support  will  be  in-house 
or  contract 

2.  Corrective  hardware/software  maintenance 

3.  Maintenance  planning  should  be  a  continuous  process 
throughout  the  life  cycle  systems 


Corrective  maintenance  involves  the  actions  performed, 
as  a  result  of  failure,  to  restore  a  system  to  an 
operational  condition  (ILS  Guide,  1986:A-1).  Adequate 
corrective  maintenance  procedures  ensured  availability  of 
automated  cartographic  or  photogrammetric  systems  to  meet 
production  requirements  for  DMA  earth  data  products. 

Maintenance  planning  required  continuous  updating 
throughout  the  life  cycles  of  automated  cartographic  or 
photogrammetric  systems.  The  maintenance  plan  developed  in 
the  detailed  design  and  development  phase  identifies 
required  levels  of  maintenance,  equipment,  and  the  intervals 
needed  to  support  a  system  (ILS  Guide,  1986:7-5).  The 
maintenance  plan  is  one  example  of  a  maintenance  document 
which  requires  updating  throughout  the  life  cycle  of  a 
system. 
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Manpower  and  Personnel .  Table  30  illustrates  the  three 


most  important  manpower  and  personnel  factors  unique  to 
automated  cartographic  or  photogrammetric  systems. 


Table  30 

Three  Most  Important  Manpower  and  Personnel  Factors 

Unique  to 

Automated  Cartographic  or  Photogrammetric  Systems 


1.  Ability  to  retain  competent  personnel 

2.  Limited  skilled  personnel  to  maintain  hardware  and 
software 

3.  Systems  need  to  be  designed  and  maintained  to  meet 
requirements  of  production  manpower  and  personnel 


The  problem  of  retaining  competent  and  skilled 
personnel  required  to  maintain  automated  cartographic  or 
photogrammetric  systems  is  a  high  priority  for  DMA. 
Appropriate  skill  levels  and  quantities  of  maintenance 
personnel  must  be  attained  to  ensure  the  required  support  of 
systems  (Finkelstein  and  Guertin,  1988:114).  Flexible  work 
schedules,  pay  adjustments,  and  fringe  benefit  incentives 
are  avenues  DMA  and  the  government  pursue  to  hire  and  retain 
skilled  maintenance  personnel.  Design  of  these  systems 
permitted  simple  preventive  maintenance  tasks  to  be 
performed  by  the  operator  of  the  system.  Daily  maintenance 
tasks  performed  by  the  operators  of  these  systems  minimize 
extended  down  times  associated  with  dust  and  dirt  build  up. 
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SuppI v  Support .  Table  31  illustrates  the  three  most 


important  supply  support  factors  unique  to  automated 
cartographic  or  photogrammetric  systems. 

Table  31 

Three  Most  Important  Supply  Support  Factors 

Unique  to 

Automated  Cartographic  or  Photogrammetric  Systems 


1.  Adequate  shelf  stock  of  required  parts 

2.  Unique  systems  need  unique  spares,  requiring  long  lead 
time 

3.  Commercial  Off-The-Shelf  (COTS)  equipment  has  impact  on 
the  provisioning  process,  proprietary  data  rights,  and 
basic  supply  support  concept 


Maintaining  adequate  levels  of  spare  parts  ensured  the 
availability  of  automated  cr  .tographic  or  photogrammetric 
systems  to  meet  DMA  mission  requirements.  Special 
components,  such  as  Integrated  Circuit  Boards  (ICB)  and 
optics,  required  long  lead  times  for  repairs.  Levels  of 
shelf  stock  for  these  components  must  be  maintained  to 
minimize  down  time. 

DMA  automated  systems,  as  well  as  many  other  government 
systems,  contain  COTS  components.  COTS  components  for  DMA 
automated  cartographic  or  photogrammetric  systems  are 
procured  directly  through  the  COTS  vendor  rather  than 
through  the  system  contractor.  Interrelationships  between 
acquisition  and  logistics  personnel  guaranteed  procedures 
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were  in  place  to  address  potential  problems  associated  with 
COTS  equipment. 

Support  Equipment .  Table  32  illustrates  the  three  most 
important  support  equipment  factors  unique  to  automated 
cartographic  or  photogrammetric  systems. 


Table  32 

Three  Host  Important  Support  Equipment  Factors 

Unique  to 

Automated  Cartographic  or  Photogrammetric  Systems 


1.  Calibration  of  support  and  test  equipment 

2.  Design  for  easily  replaced  logical  units,  which  are 
easily  repaired 

3.  Calibration  of  photogrammetric  equipment  is  critical 


The  experts'  comments  focused  on  the  calibration  of 
unique  support  equipment.  Precise  cartographic  or 
photogrammetric  specifications  required  calibrated  support 
and  test  equipment  be  maintained  on-site  at  the  production 
centers.  Calibration  of  the  support  and  test  equipment  was 
performed  by  contractors  through  an  inter-service  support 
agreement . 

Technical  Data .  Table  33  illustrates  the  three  mo3t 
important  technical  data  factors  unique  to  automated 
cartographic  or  photogrammetric  systems. 

Technical  data  allowed  maintenance  personnel  to  perform 
corrective  maintenance  such  as  fault  isolation  and 
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Table  33 


Three  Most  Important  Technical  Data  Factors 

Unique  to 

Automated  Cartographic  or  Photogrammetric  Systems 


1.  Maintenance  and  diagnostics  documentation 

2.  Software  documentation  and  interface  manuals 

3.  Communicate  with  contractor  when  developing  manuals  used 
for  production  and  maintenance 


troubleshooting.  Hardware  and  software  maintenance 
documentation  developed  by  the  systems'  contractors  were 
provided  to  DMA  at  time  of  delivery.  DMA  personnel  assisted 
in  the  design  and  development  of  these  documents  during  the 
advanced  development  phase  (Finkelstein  and  Guertin, 
1988:136).  Formal  review  of  the  documents  upon  delivery  to 
DMA  assured  compliance  with  requirements  set  forth  early  in 
the  life  cycle. 

Training .  Table  34  illustrates  the  three  most 
important  training  factors  unique  to  automated  cartographic 
or  photogrammetric  systems. 

The  training  for  DMA  automated  cartographic  or 
photogrammetric  systems  included  both  in-house  and 
contractor  conducted  courses.  Before  training  commenced, 
training  documents  were  reviewed  to  correct  discrepancies. 
Cadre  training  conducted  by  systems'  contractors  provided 
DMA  with  in-house  personnel  who  could  assist  with 
preparatory  training.  Preparatory  training  furnished 


Table  34 


Three  Most  Important  Training  Factors 
Unique  to 

Automated  Cartographic  or  Photogrammetric  Systems 


1.  Critical/preliminary  design  review  of  training  plans 

2.  In  house  hardware/software  training  must  be  in  depth 

3.  Need  preparatory  training 


employees  with  fundamental  skills  required  for  detailed 
management,  maintenance,  and  operations  training  for 
automated  cartographic  or  photogrammetric  systems.  Once 
systems  were  operational ,  additional  in-house  training 
provided  new  employees  with  the  skills  required  to  support 
DMA  systems. 

Computer  Resources  Support.  Table  35  illustrates  the 
three  most  important  computer  resources  support  factors 
unique  to  automated  cartographic  or  photogrammetric  systems. 

The  computer  resources  support  element  pertained  to  the 
hardware  and  software  resources  required  to  maintain  the 
automated  cartographic  or  photogrammetric  systems.  Growing 
concern  from  DMA  experts  centered  on  diagnostic  software 
required  to  trace  hardware  deficiencies  and  fault  isolation 
to  a  line  replaceable  unit.  Lessons  learned  from  MARK  85 
demonstrated  a  need  to  maintain  a  separate  facility 
to  test  upgrades  to  software  codes  without  impacting 
operational  systems  and  delaying  production  schedules. 
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Table  35 


Three  Most  Important  Computer  Resources  Support  Factors 

Unique  to 

Automated  Cartographic  or  Photogrammetric  Systems 


1.  Diagnostic  software 

2.  Flexibility,  expandability,  peak  capacity,  and  fault 
isolation  to  Line  Replaceable  Unit  (LRU) 

3.  Need  a  development  test  facility  to  test  software  code 
fixes  and  upgrades  instead  of  tying  up  production 
equipment 


Faci 1 ities .  Table  36  illustrates  the  three  most 
important  facilities  factors  unique  to  automated 
cartographic  or  photogrammetric  systems. 


Table  36 

Three  Most  Important  Facilities  Factors 
Unique  to 

Automated  Cartographic  or  Photogrammetric  Systems 


1.  Designed  for  human  occupation  not  just  machines 

2.  Clean  environment 

3.  Space  requirements 


Unlike  many  automated  systems,  DMA  automated  * 

cartographic  or  photogrammetric  systems  required  a  human 
interface  for  the  entire  period  of  operation.  Design 
engineering  and  logistics  support  requirements  must  take 
into  account  ergonomic  considerations.  As  with  any 
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automated  system,  DMA  systems  are  operated  in  a  clean 
environment  to  minimize  the  risk  associated  with  dust  and 
dirt.  Space  requirements  were  a  concern  although  the 
construction  of  new  facilities  at  the  DMA  Reston  Center  and 
DMA  Hydrographic/Topographic  Center  alleviated  some  space 
problems. 

Packaging .  Handling .  Storage .  and  Transportation. 

Table  37  illustrates  the  three  most  important  packaging, 
handling,  storage,  and  transportation  factors  unique  to 
automated  cartographic  or  photogrammetric  systems. 

Table  37 

Three  Most  Important 

Packaging,  Handling,  Storage,  and  Transportation  Factors 

Unique  to 

Automated  Cartographic  or  Photogrammetric  Systems 


1.  Security  of  classified  items 

2.  Equipment  size  and  weight 

3.  Shipment  of  component  items  from  repair  depot  to 
customer 


Security  procedures  are  always  the  first  consideration 
for  all  DMA  activities.  Security  control  methods  for 
»  documents,  software,  equipment  and  components  must  be 

considered  early  in  the  packaging,  handling,  storage,  and 
transportation  phase  of  the  system  life  cycle.  The  size  and 
weight  of  automated  cartographic  or  photogrammetric  systems 
required  planning  to  allow  for  restricted  passageways. 
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elevators,  raised-floor  load  capacities,  and  handling 
equipment . 

Design  Interface .  Table  38  illustrates  the  three  most 
important  design  interface  factors  unique  to  automated 
cartographic  or  photogrammetric  systems. 

Table  38 

Three  Most  Important  Design  Interface  Factors 

Unique  to 

Automated  Cartographic  or  Photogrammetric  Systems 


1.  Function  of  overall  system  design 

2.  Reliability/Maintainability  requirements 

3.  Need  to  incorporate  this  element  early  in  the  life  cycle 


System  readiness  objectives  and  logistics  constraints 
associated  with  design  interface  are  established  early  in 
the  systems  life  cycle.  The  major  functions  of  design 
interface  were  reliability  and  maintainability  as  defined  by 
the  DMA  experts.  Reliability  designed  early  in  the  life 
cycle  of  a  system  adds  to  the  supportability  of  the  system. 
Maintainability  applied  early  in  the  life  cycle  prevents 
high  maintenance  costs  during  system  operations. 

These  unique  factors  of  ILS  elements  provided  guidance 
for  identifying  logistics  support  planning  requirements 
applicable  to  automated  cartographic  or  photogrammetric 
systems . 
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Research  Question  Three: 


At  what  stages  of  the  automated  system's  life  cycle  do 

the  ILS  elements  require  determination  and  design? 

The  Delphi  survey  required  experts  to  complete 
questions  concerning  when  in  the  life  cycle  of  automated 
cartographic  or  photogrammetric  systems  should  determination 
and  design  requirements  for  each  ILS  element  take  place. 

The  Delphi  experts  reached  consensus  on  eight  of  the  ten  ILS 
elements.  The  two  ILS  elements  for  which  consensus  was  not 
reached  were  computer  resources  support  and  facilities. 

Using  the  mean  values  depicted  in  Table  28,  page  91, 
determination  and  design  requirements  for  computer  resources 
support  should  take  place  during  the  advanced  development 
phase,  and  facilities  during  the  detailed  design  and 
development  stage.  Figure  16  illustrates  at  what  phase  of 
the  automated  cartographic  or  photogrammetric  systems  life 
cycle  ILS  requirements  determination  and  design  take  place. 
Delphi  results  indicated  maintenance  planning,  computer 
resources  support,  and  design  interface  determination  and 
design  requirements  take  place  during  the  advanced 
development  stage.  Manpower  and  personnel,  supply  support, 
support  equipment,  technical  data,  training,  facilities,  and 
packaging,  handling,  storage,  and  transportation 
determination  and  design  requirements  take  place  during  the 
detailed  design  and  development  stage.  In  the  following 
section,  the  Delphi  responses  are  analyzed  with  respect  to 
the  findings  of  the  literature  review. 
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Figure  16.  Life  Cycle  Determination  and  Design  Requirements 
for  ILS  Elements 
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Maintenance  Planning.  The  Delphi  responses  for 
maintenance  planning  agreed  with  those  presented  in  the 
literature  review.  Although  the  literature  indicated  the 
maintenance  concept  is  developed  during  the  conceptual 
phase,  the  specific  maintenance  tasks  and  operational 
requirements  are  not  identified  until  the  advanced 
development  phase.  Furthermore,  the  detailed 
maintenance  plan  is  not  created  until  the  end  of  the 
detailed  design  and  development  phase  or  early  in  the 
production  phase  (Finkelstein  and  Guertin,  1988:110). 

Delphi  results  indicated  maintenance  planning  determination 
and  design  requirements  occur  during  the  advanced 
development  phase.  This  allowed  adequate  time  for  the 
development  of  maintenance  requirements  for  automated 
cartographic  or  photogrammetric  systems. 

Manpower  and  Personnel .  The  literature  discussed  the 
need  to  first  estimate  manpower  and  personnel  criteria 
during  the  conceptual  phase.  Precise  manpower  and  personnel 
requirements  were  formulated  in  accordance  with  the  Delphi 
results.  Both  the  literature  review  and  the  Delphi  results 
indicated  specific  manpower  and  personnel  determination  and 
design  requirements  occur  during  the  detailed  design  and 
development  phase  of  automated  cartographic  or 
photogrammetric  systems.  Updates  to  manpower  and  personnel 
are  made  according  to  production  requirements  throughout  the 
remaining  life  cycle. 


SuppI y  Support .  According  to  the  literature,  supply 
support  is  initiated  during  the  advanced  development  phase. 

The  Delphi  results  indicated  supply  support  determination 
and  design  requirements  occur  during  the  detailed  design  and 

* 

development  phase.  Although  provisioning  occurs  during  the 
detailed  design  and  development  phase,  planning  for  parts 
obsolescence  must  take  place  during  the  advanced  development 
phase. 

Support  Equipment .  The  literature  indicated  the 

support  equipment  element  must  be  planned  and  developed 

during  the  conceptual  phase.  The  decision  whether  to 

purchase  new  support  equipment  or  use  existing  equipment 

must  take  place  early  in  the  system  life  cycle.  The  Delphi 

responses  agreed  with  specific  support  equipment 

considerations  presented  in  the  literature. 

During  the  detailed  design  and  development  phase, 
detailed  task  analyses  and  documentation  are  performed 
to  identify  the  specific  equipment  requirements  for 
every  operating  and  maintenance  task.  (ILS  Guide, 

1986:7-9). 

Technical  Data .  The  literature  disagreed  with  the 
Delphi  results  for  the  technical  data  element.  The 
literature  stated  that  technical  data  requirements  take 
place  in  the  advanced  development  phase  and  the  Delphi 

4 

results  indicated  the  detailed  design  and  development  phase. 

Both  technical  data,  relating  to  operations  and  maintenance, 

♦ 

and  management  data,  must  be  developed  during  the  advanced 
development  phase  (Finkelstein  and  Guertin,  1988:136). 

Preliminary  technical  data  manuals  must  be  accessible  late 
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in  the  advanced  development  phase  to  support  testing  and 
training  functions  (ILS  Guide,  1986:7-10).  These  findings 
supported  the  need  to  determine  and  design  technical  data 
requirements  in  the  advanced  development  phase. 

Training .  The  Delphi  results  for  the  training  element 
agreed  with  the  literature  review.  The  Delphi  results  and 
the  literature  review  indicated  requirements  for  training 
determination  and  design  during  the  detailed  design  and 
development  phase. 

Although  DODD  5000.39  specifies  that  detailed 
descriptions  of  current  and  projected  skill  and 
training  resources  be  developed  during  the  conceptual 
phase,  specific  requirements  for  training  and  equipment 
are  performed  during  advanced  development  and  detailed 
design  and  development  phases.  (ILS  Guide,  1986:7-10) 

Computer  Resources  Support .  The  Delphi  results 

supported  the  requirement  for  determination  and  design  of 

computer  resources  support  during  the  advanced  development 

stage.  The  literature  review  presented  the  same  findings. 

During  the  conceptual  phase  and  the  advanced  development 

phase  system  concepts  are  developed  and  the  system/hardware 

and  system/software  requirements  are  defined  and  examined 

(Tile,  1987:238). 

Facilities .  Consensus  was  not  reached  for  this  element 
during  the  Delphi  survey.  Although  the  mean  value  indicated 
the  selection  was  detailed  design  and  development,  40%  of 
the  experts  chose  the  advanced  development  phase.  The 
literature  indicated  the  facilities  planning  requirements 
function  takes  place  during  the  advanced  development  phase. 


Ninety-six  percent  of  the  Delphi  experts  selected  either  the 
advanced  development  or  detailed  design  and  development 
phase.  The  results  tended  to  support  the  findings  of  the 
literature  review. 

Packaging .  Handling .  Storage .  and  Transportation.  The 
literature  review  and  Delphi  results  disagreed  with  respect 
to  determination  and  design  requirements  for  packaging, 
handling,  storage,  and  transportation.  The  literature 
review  supported  the  advanced  development  phase,  while  the 
Delphi  results  advocated  the  detailed  design  and  development 
phase.  During  the  advanced  development  phase,  the 
literature  indicated  that  specific  transportation  attributes 
are  identified,  and  packaging  and  handling  requirements  are 
evaluated  and  determined  (ILS  Guide,  1986:7-11).  The 
findings  supported  the  advanced  development  phase  for 
requirements  determination  and  design  of  the  packaging, 
handling,  storage,  and  transportation  element. 

Design  Interface .  The  literature  review  and  the  Delphi 
results  agreed  on  determination  and  design  requirements  for 
the  design  interface  element.  The  two  major  functions  of 
design  interface,  reliability  and  maintainability,  must  be 
determined  and  designed  in  the  advanced  development  phase. 
Early  life  cycle  design  of  these  functions  ensures 
supportabi 1 ity  and  prevents  high  maintenance  costs 
throughout  the  entire  system  life  cycle  (Finkelstein  and 
Guertin,  1988:72). 
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Conclusion 


This  chapter  analyzed  the  Delphi  results  found  in 
Chapter  IV.  Where  possible,  the  results  of  the  Delphi 
survey  were  compared  against  those  findings  presented  in  the 
literature  review  found  in  Chapter  II.  A  descriptive  model 
was  developed  based  on  the  results  of  the  Delphi  survey  of 
31  DMA  experts.  A  list  of  the  three  most  important  factors, 
for  each  ILS  element,  unique  to  automated  cartographic  or 
photogrammetric  systems  was  collated  from  the  Delphi 
results.  Finally,  results  from  the  Delphi  survey  were 
compared  with  the  literature  to  determine  when  in  the  life 
cycle  of  automated  cartographic  or  photogrammetric  systems 
determination  and  design  requirements  for  each  ILS  element 
should  take  place.  The  answers  to  the  research  questions 
found  in  this  chapter  provided  the  framework  required  to 
identify  logistics  support  planning  requirements  for 
automated  cartographic  or  photogrammetric  systems.  Chapter 
VI  presents  the  contributions  of  this  research  and 
recommendations  for  future  research. 
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VI .  Conclusions  and  Recomm»  ndations 

0 

Today's  high-technology  military  weapon  systems  require 
precise  digital  mapping,  charting,  and  geodetic  data.  To 
respond  to  these  digital -based  earth  data  requirements,  DMA 
initiated  a  massive  program  to  convert  from  a  manual  to  an 
all  digital  production  capability  by  1993.  Requirements  for 
accurate  and  timely  digital -based  earth  data  products 
necessitated  proper  logistics  support  planning  for  automated 
cartographic  or  photogrammetric  production  systems. 

Although  DMA  lacked  an  integrated  approach  of 
formulating  logistical  support  planning  requirements  for 
their  automated  cartographic  or  photogrammetric  systems, 
management  recognized  the  importance  of  the  Integrated 
Logistics  Support  (ILS)  concept.  Both  the  creation  of  a  new 
Directorate  for  Acquisition,  Installation,  and  Logistics  and 
the  sponsorship  of  this  research  support  the  premise  that 
DMA  has  identified  the  need  for  logistics  support  planning 
of  automated  cartographic  or  photogrammetric  systems. 

This  research  was  initiated  to  determine  how  an 
Integrated  Logistics  Support  (ILS)  or  other  logistical 
support  planning  model  can  provide  a  means  for  determining  « 

logistical  support  requirements  for  DMA  automated  systems. 

The  literature  review  identified  Ostrofsky’s  Logistics 

% 

Systems  Management  Matrix  (LSMM)  as  the  basis  for  proper 
implementation  of  a  logistics  support  plan  for  any  given 
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system  (Ostrofsky,  1986:8).  The  LSMM  model  was  used  as  the 
founda.ion  for  the  development  of  a  descriptive  model 
applicable  ti  logistics  support  planning  of  automated 
cartographic  or  photogrammetric  systems. 

The  Delphi  method  provided  the  means  for  constructing  a 
model  by  polling  expert  DMA  administrative,  research  and 
development,  support  and  maintenance,  and  production 
personnel.  The  intent  of  this  methodology  was  to  reach  a 
level  of  consensus  on  characteristics  and  attributes  of  a 
model  applicable  to  automated  cartographic  or 
photogrammetric  systems.  The  validity  of  the  model 
constructed  was  ensured  by  the  participation  of  DMA  experts 
using  the  iterative  Delphi  process.  The  survey  results 
concluded  DMA  experts  agreed  chat  the  LSMM  model  provided 
the  basic  structure  for  determining  logistics  support 
procedures  for  automated  cartographic  or  photogrammetric 
systems  employed  by  DMA.  The  experts  provided  feedback  on 
what  factors,  within  each  of  the  ILS  elements  portrayed  in 
the  LSMM  model,  were  unique  to  logistics  support  planning 
for  automated  cartographic  or  photogrammetric  systems. 
Finally,  the  experts  decided  at  what  phase  of  the  automated 
cartographic  or  photogrammetric  system  life  cycle  the  ILS 
elements  required  determination  and  design. 

Discussion 

This  research  was  initiated  to  determine  how  a 
logistics  support  model  can  provide  a  framework  for 


119 


determining  logistical  support  requirements  for  DMA 
automated  cartographic  or  photogrammetric  systems.  The 
research  objectives  to  accomplish  this  were:  (1)  to  develop 
a  descriptive  model  to  address  the  logistical  support 
requirements  for  automated  cartographic  or  photogrammetric 
systems;  and  (2)  validate  the  model  by  determining  its 
relevance  to  DMA  automated  cartographic  or  photogrammetric 
systems . 

The  descriptive  model  is  a  very  general  guide  to 
determining  logistics  support  requirements  for  automated 
cartographic  or  photogrammetric  systems.  The  model  provided 
the  basic  principles  of  management,  the  design  or  life  cycle 
of  systems,  and  the  ILS  elements  required  to  support 
automated  cartographic  or  photogrammetric  systems.  Another 
characteristic  of  the  descriptive  model  is  its  similarity  to 
the  Logistics  Systems  Management  Matrix  (LSMM).  The 
addition  of  the  ILS  security  element  is  the  only  difference 
between  the  models.  This  is  not  unusual  as  the  LSMM  forms 
the  basis  for  proper  implementation  of  a  logistics  support 
plan  for  any  given  system.  The  modifications  to  Ostrofsky’s 
original  LSMM  provided  a  common  basis  of  understanding  for 
the  Delphi  experts. 

The  descriptive  model  developed  was  validated  by  using 
the  iterative  Delphi  process.  The  survey  process  determined 
if  DMA  management  believed  the  characteristics  and 
attributes  of  the  model  were  valid  for  DMA's  high- 


technology,  fast-obsolescence  systems.  Feedback  from  the 
experts  confirmed  the  validity  of  the  descriptive  model. 

Conclusions 

Although  descriptive  in  nature,  this  research  provided 
the  framework  required  to  identify  logistics  support 
planning  requirements  for  automated  cartographic  or 
photogrammetric  systems.  The  following  six  conclusions 
summarize  the  findings  of  this  research. 

1.  The  need  existed  for  a  structured,  integrated 
logistics  support  approach  to  automated  cartographic  or 
photogrammetric  systems  planning.  Results  from  the  Delphi 
survey  revealed  DMA  only  employs  ILS  concepts  to  a  moderate 
extent.  Nevertheless,  DMA  experts  recognized  the  importance 
of  ILS  for  automated  cartographic  or  photogrammetric  systems 
planning.  The  experts  reached  a  level  of  consensus  (94% 
agreement)  that  understanding  and  applying  ILS  concepts  was 
important  for  logistics  support  planning  of  automated 
cartographic  or  photogrammetric  systems. 

2.  Beyond  an  ILS  approach,  the  need  existed  for  an 
integrated  organizational  structure  permitting  the  execution 
of  an  ILS  plan.  The  functional  differences  of  DMA  experts 
prevented  consensus  for  several  items  in  the  Delphi  survey. 
The  experts  were  unable  to  reach  consensus  for  the  ranking 
of  design  or  life  cycle  phases  and  ILS  elements.  The 
findings  in  this  research  supported  the  findings  of  the  DOD 
and  DMA  Inspector  Generals. 


In  the  past,  logistics  functions  were  performed  in  a 

variety  of  DMA  components.  The  DOD  Inspector  General  (IG) 

Report  of  1989  recommended  the  consolidation  of  DMA 

logistics  functions  into  a  single  unified  directorate  (DOD 

IG  Report  INS  89-02A,  1989:47).  The  DMA  IG  supported  the 

DOD  IG  findings  and  stated 

...  the  DMA  IG  recommends  the  consolidation  of  all 
logistics  and  logistics-related  functions  as  a  Deputate 
for  logistics  and  engineering  to  include  directorates 
for  logistics  plans,  maintenance,  transportation, 
supply  and  distribution,  procurement,  and  facilities. 
(DMA  IG  Report,  DMASC  Inspection  Report,  1989:111-1) 

This  high-level  integrated  approach  to  logistics  is  not 

unique  to  DMA  or  DOD  and  is  evident  in  competitive 

commercial  firms. 

In  successful  firms  logistics  has  been  elevated  to  a 
top  management  position,  often  of  equal  ranking  with 
marketing  (requirements)  and  production.  One  study 
fpund  that  of  47  companies  having  a  physical 
distribution  (logistics)  manager,  in  37  (79  percent)  of 
the  companies  he/she  was  either  a  vice  president  or 
reported  to  a  vice  president.  (Ballou,  1973:8) 

3.  The  need  existed  to  determine  and  design  logistics 

support  requirements  early  in  the  system  life  cycle  of 

automated  cartographic  or  photogrammetric  systems.  The 

Delphi  results  indicated  maintenance  planning,  computer 

resources  support,  and  design  interface  determination  and 

design  requirements  take  place  during  the  advanced 

development  stage,  while  manpower  and  personnel,  supply 

support,  support  equipment,  technical  data,  training, 

facilities,  and  packaging,  handling,  storage,  and 

transportation  determination  and  design  requirements  take 


place  during  the  detailed  design  and  development  stage. 

These  findings  agreed  with  the  results  of  ranking  the  design 
or  life  cycle  phases.  The  experts  ranked  detailed  design 
and  development  phase  first  and  advanced  development  phase 
second. 

4.  Maintenance  planning  was  most  vital  to  the  ILS 
planning  effort  of  automated  cartographic  or  photogrammetric 
systems.  Both  the  literature  and  the  Delphi  results 
supported  this  finding.  Maintenance  planning  is  both  the 
first  of  ten  elements  a  logistics  manager  must  evaluate  and 
"the  process  to  develop  maintenance  concepts  and  maintenance 
requirements  of  the  system"  (ILS  Guide,  1986:7-5).  The 
experts  also  reached  consensus  and  ranked  maintenance 
planning  first,  out  of  the  ten  ILS  elements,  in  terms  of 
importance  for  logistics  support  planning  of  automated 
cartographic  or  photogrammetric  systems. 

5.  While  important,  packaging,  handling,  storage,  and 
transportation  (PHS&T)  was  least  vital  to  the  ILS  planning 
effort  of  automated  cartographic  or  photogrammetric  systems. 
The  experts  reached  consensus  and  ranked  (PHS&T)  tenth,  out 
of  the  ten  ILS  elements,  in  terms  of  importance  for 
logistics  support  planning  of  automated  cartographic  or 
photogrammetric  systems.  This  was  attributable  in  part  to 
the  absence  of  delivery  problems  in  the  recent  deployment  of 
various  automated  systems.  Also,  some  experts  provided 
open-ended  comments  which  described  automated  cartographic 
or  photogrammetric  systems  as  relatively  static.  Because 


these  systems  were  moved  infrequently,  packaging,  handling, 
storage,  and  transportation  was  considered  the  least 
important  ILS  element. 

6.  DMA  logistics  personnel  without  cartographic  or 
photogrammetric  backgrounds  should  use  the  identified  unique 
factors  within  each  ILS  element  to  gain  insight  to  the 
special  logistics  requirements  of  automated  cartographic  or 
photogrammetric  systems.  Particular  areas  requiring 
attention  were:  (1)  design  and  maintenance  requirements  to 
meet  ergonomic  needs  of  cartographic  or  photogrammetric 
production  personnel;  (2)  calibration  requirements  of 
photogrammetric  equipment;  (3)  development  a  test  facility 
to  verify  software  code  fixes  and  upgrades  instead  of  tying 
up  production  systems;  and  (4)  meet  security  requirements 
for  all  logistics  support  items. 

Recommendations 

The  following  four  recommendations  for  future  research 
and  use  of  the  descriptive  model  are  suggested. 

1.  Future  research  based  on  the  descriptive  model 
developed  in  this  thesis  should  utilize  DOD  life  cycle 
terminology.  At  the  time  this  research  was  conducted,  DOD 
life  cycle  terminology  was  under  revision.  Rather  than 
introduce  confusion  brought  about  by  changing  DOD 
terminology,  Finkelstein  and  Guertin's  life  cycle 
terminology  was  used.  This  terminology  provided  a  basis  for 
common  understanding  of  life  cycle  phases. 
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2.  Research  should  be  conducted  to  examine  the 


principles  of  management  as  they  apply  to  the  other  two 
dimensions  of  the  descriptive  model.  The  limited  treatment 
of  the  principles  of  management  in  this  research  was  a 
function  of  the  time  constraint  inherent  to  the  thesis 
effort. 

3.  DMA  management  should  consider  employing  the 
findings  of  this  research  because  it  was  intended  to  be  used 
by  managers  for  generalized  logistics  support  requirements 
planning  of  automated  cartographic  or  photogrammetric 
systems.  The  descriptive  model  should  become  part  of 
initial  DMA  logistics  support  planning  activities  for 
automated  cartographic  or  photogrammetric  systems.  The 
descriptive  model  will  provide  DMA  logistics  managers,  as 
well  as  other  functional  managers,  with  guideline  criteria 
for  developing  specific  logistics  support  requirements  for 
automated  cartographic  or  photogrammetric  systems.  It  will 
ensure  the  integration  of  management  principles  and  ILS 
elements  throughout  the  entire  life  cycle  of  automated 
cartographic  or  photogrammetric  systems. 

4.  In  addition  to  automated  cartographic  or 
photogrammetric  systems,  DMA  utilizes  various  automated 
management  information  and  photographic  systems.  The  need 
exists  for  exploratory  ILS  research  for  these  automated 
systems.  The  findings  of  this  research  provide  a  basis  for 
logistics  support  applications  on  any  automated  systems 
employed  in  or  outside  of  DMA. 


Contributions 


The  four  contributions  of  this  research  are:  (1)  the 
descriptive  model;  (2)  the  unique  factors,  within  each  ILS 
element,  applicable  to  automated  cartographic  or 
photogrammetric  systems;  (3)  life  cycle  determination  and 
design  requirements  for  each  ILS  element;  and  (4)  the 
comments  provided  by  the  DMA  Delphi  experts. 

Descriptive  Model .  The  descriptive  model  developed  in 
this  research  effort  provides  DMA  managers  a  logistics 
support  planning  tool  applicable  to  automated  cartographic 
or  photogrammetric  systems.  The  model  represents  a  combined 
feedback  effort  from  various  functional  DMA  experts.  These 
experts  possess  a  wealth  of  knowledge  and  experience 
concerning  automated  cartographic  or  photogrammetric 
systems.  The  Delphi  method  provided  the  means  required  to 
capture  the  opinions  of  these  experts  and  formulate  the 
findings  into  a  descriptive  model.  The  experts'  interest  in 
the  subject  is  reflected  by  the  amount  of  opinion  data 
provided  in  the  two  rounds  of  questioning. 

The  DMA  experts  reached  consensus  on  the  dimensions  of 
the  descriptive  model  because  the  model  accurately  portrayed 
the  characteristics  and  attributes  required  for  logistics 
support  planning  of  automated  cartographic  or 
photogrammetric  systems.  The  model  incorporated  the 
management  principles,  design  or  life  cycle  phases,  and  ILS 
elements  required  to  support  DMA  automated  cartographic  or 
photogrammetric  systems.  Although  the  Delphi  respondents 


possessed  diverse  functional  expertise  within  DMA,  their 
ability  to  understand  and  reach  consensus  on  the  dimensions 
of  the  descriptive  model  verified  the  future  use  of  the 
model  for  DMA  automated  cartographic  or  photogrammetric 
systems . 

Total  consensus  was  not  reached  for  the  ranking  of 
design  or  life  cycle  phases  and  ILS  elements.  Non-consensus 
for  certain  design  or  life  cycle  phases  resulted  from 
functional  bias  of  the  experts.  Production  experts  believed 
the  operational  use  phase  was  most  important  while  research 
and  development  and  support  and  maintenance  experts 
concluded  the  advanced  development  phase  was  most  important. 

Non-consensus  and  inconsistencies  for  the  ILS  elements 
resulted  from  the  lack  of  total  understanding  of  ILS  element 
definitions.  Although  detailed  definitions  were  provided  to 
the  experts  in  both  rounds  of  Delphi  questioning,  the 
feedback  from  the  experts  presented  a  lack  of  experience  in 
applying  ILS  concepts. 

Unique  Factors  Within  ILS  Elements .  The  lists  of 
unique  factors,  within  each  ILS  element,  applicable  to 
automated  cartographic  or  photogrammetric  systems  provided  a 
general  reference  for  planning,  designing,  and  developing 
logistics  support  procedures  for  automated  cartographic  or 
photogrammetric  systems. 

The  Delphi  responses  provided  by  the  DMA  experts 
represent  professional  opinions  from  some  of  the  foremost 
minds  in  cartography  or  photogrammetry .  Many  logistics 


managers  employed  by  DMA  do  not  possess  cartographic  or 
photogrammetric  skills  or  education.  The  lists  of  unique 
factors,  within  each  ILS  element,  applicable  to  automated 
cartographic  or  photogrammetric  systems  provide  the 
beginnings  of  an  expert  system  for  logistics  managers  not 
possessing  cartographic  or  photogrammetric  skills.  These 
lists  are  also  a  valuable  tool  for  other  governmental 
agencies  employing  automated  cartographic  or  photogrammetric 
systems.  Finally,  some  of  the  factors  are  generic  in  nature 
and  can  be  applied  to  other  automated  management  information 
systems . 

Life  Cycl e  Determination  and  Design  Requirements . 

The  life  cycle  determination  and  design  requirements  for 
each  ILS  element  provide  managers  an  excellent  tool  for 
logistics  support  planning  purposes.  These  findings 
represent  generalized  descriptions  of  specifically  outlined 
logistics  support  determination  and  design  tasks  found  in 
DODD  5000.39. 

The  Delphi  results  agreed  with  the  literature  review 
for  determination  and  design  requirements  of  automated 
cartographic  or  photogrammetric  systems.  The  experts  agreed 
all  determination  and  design  requirements  for  ILS  elements 
should  take  place  early  in  the  system  life  cycle.  These 
findings  impact  the  logistics  support  planning  requirements 
for  the  entire  life  cycle  of  automated  cartographic  or 
photogrammetric  systems.  The  design,  development,  and 
implementation  of  automated  cartographic  or  photogrammetric 
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systems  involve  interrelationships  of  the  four  functional 
groups  which  participated  in  the  Delphi  survey.  Agreement 
by  the  four  functional  groups  on  the  need  for  early 
determination  and  design  of  ILS  elements  will  allow  DMA  to 
design  supportability  into  automated  cartographic  or 
photogrammetric  systems.  In  turn,  early  determination  and 
design  will  lower  operations  and  maintenance  costs  and 
provide  increased  levels  of  mission  readiness. 

Delphi  Experts '  Comments .  The  final  contribution  of 
this  research  is  the  opinion  data  provided  by  DMA  experts. 
The  experts  reached  consensus  and  determined  DMA  employs  ILS 
to  its  automated  cartographic  or  photogrammetric  systems 
only  to  a  moderate  extent.  The  experts'  comments  further 
suggest  a  need  for  an  Integrated  Logistics  Support  approach 
for  DMA  automated  cartographic  or  photogrammetric  systems. 

The  creation  of  a  Directorate  for  Acquisition, 
Installation,  and  Logistics  must  integrate  the  fragmented 
logistics  support  policies  and  procedures  in  place  at  DMA. 

To  minimize  the  support  and  maintenance  costs  associated 
with  operational  systems,  logistics  support  requirements 
must  be  designed  and  developed  early  in  the  life  cycle  of 
automated  cartographic  or  photogrammetric  systems. 

Summary 

Government  agencies  and  military  services  require 
accurate  and  timely  earth  data  (mapping,  charting,  and 
geodetic  data)  to  support  mission  objectives.  The  Defense 
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Mapping  Agency  is  responsible  for  providing  earth  data  in 
digital  and  graphic  formats  through  the  use  of  automated 
cartographic  or  photogrammetric  systems.  It  is  imperative 
that  DMA  adapt  an  Integrated  Logistics  Support  (ILS) 
approach  to  properly  support  and  maintain  these  automated 
systems . 

This  research  provided  a  thorough  description  of 
logistics  support  planning  for  DMA  automated  cartographic  or 
photogrammetric  systems.  Its  analysis  demonstrated  the  need 
for  an  integrated  structured  approach  to  logistics  support, 
an  integrated  organizational  structure  to  implement  ILS 
plans,  and  an  increased  emphasis  on  maintenance  planning  to 
more  effectively  support  automated  cartographic  or 
photogrammetric  systems. 
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Appendix  A:  Weapons  Systems  Requiring  DMA  Earth  Data 


Joint  Service  Systems 

AGM-86  Air  Launched  Cruise  Missile  (ALCM) 

AGM-129  Advanced  Cruise  Missile  (ACM) 

Cruise  Missile  Advanced  Guidance  (CMAG) 

DARPA  Image  Generation  (Side  VU) 

Joint  Automated  Terminal  Instrument  Procedures  (AUTO  TERPS) 
Joint  Surveillance  Target  Attack  Radar  System 
Navstar  Global  Positioning  System  (GPS) 

Precision  Location  Strike  System  (PLSS) 

Satellite  (Overland)  Altimetry 

Tactical  Reconnaissance  Exploitation  Demonstration  System 
(TREDS) /TR-1  Ground  Station  (TRIGS) 

Terminal  Fix  Sensor  (TFS) 

TR-1  Reconnaissance  System 

Tomahawk  BGM-109A/B/C/D  Cruise  Missile 


Air  Force  Systems 

Advanced  Computer  Flight  Plan  (USAFACFF) 

Advanced  Technology  Bomber 

AF  Rescue  Coordination  Center  Planning  System  (AFRCCPS) 
Automated  Combat  Mission  Folder  System  ( ACMFS) 

B-1B 

B-52  Weapon  System  Trainer  (WST) 

C-17 

C-130  Weapon  System  Trainer  (WST) 

Computer  Aided  Mission  Planning  System  (CAMPS) 

EF-11A  Operational  Flight  Trainer  (OFT)  System 
EF/F/FB-111  Weapon  System  Trainer  (WST) 

Enemy  Situation  Correlation  Element  (ENSCE) 

F-15E  Weapons  System  Trainer  (WST) 

F-16  Digital  Radar  Landmass  Simulator  (DRLMS) 

F-16  Improved  Digital  Radar  Landmass  Simulator  (IDRLMS) 
Filmstrips 

Integrated  Terrain  Access  and  Retrieval  System  (ITARS) 
Minuteman 

National  Aerospace  Plane  (NASP) 

On  Board  Electronic  Warfare  Simulator  (OBEWS) 

Pave  Pillar 
PEACEKEEPER 

Remote  Map  Readers  (RMR) 

Small  Intercontinental  Ballistic  Missile  (SICBM) 
Survivable  Strategic  Missile  Launcher 
Tactical  Air  Forces  (TAF)  Small  Computer  Project 
Tactical  Re-entry  Impacting  Munitions  (TRIM) 
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Army  Systems 


All  Source  Analysis  System  (ASAS) 

All  Source  Exploitation  In  A  Combat  Environment 
Army  Training  Battlefield  Simulation  System  (ARTBASS) 
Battlefield  Management  System  (BMS) 

Digital  Topographic  Support  System  (DTSS) 

Fiber  Optics  Guided  Missile  (FOG-M) 

Firefinder:  AN/TPQ-36  and  AN/TPQ-37  Mortar  and  Artiiler 
Locating  Radar 

Maneuver  Control  System  (MCS) 

Night  Navigational  and  Pilotage  System  (NNAPS) 

PATRIOT  Surface  to  Air  Missile  System 
Pershing  II 

Position  Locating  Reporting  System  (PLRS) 

Remotely  Monitored  Battlefield  Sensor  (REMBASS) 

Remotely  Piloted  Vehicle  (RPV-AQUILA) 

Vehicle  Integrated  Intelligence  VCINT)^ 

Vehicle  Navigation  Aid  System  (VNAS) 


Navy  Systems 

A-6E  Weapon  System  Trainer  (WST) 

AEGIS  Air  Defense  System 

AN/BSY-1,  FY89  Combat  System  (formerly  SUBACS) 

AN/WLQ-4,  Sea  Nymph 
AV-8B  Support 

Carrier  Battle  Group  (CVBG)  Special  Chart 
E-2C  Weapon  System  Trainer  (WST) 

EA-63  Weapon  System  Trainer  (WST) 

Encapsulated  Torpedo  (CAPTOR)  Mine 
Navy  Command  and  Control  System 

Navy  Optimum  Path  Aircraft  Routing  System  (OPARS) 
Over-the-Horizon-Targeting  (OTH-T)  Tomahawk  and  Harpoon 
Systems 

Trident  and  Trident  II 


Source.  (Larson  and  Pelletiere,  1989:103-05) 


Appendix  B:  MARK  85  SYSTEM 


General  Information 

MARK  85:  The  first  phase  of  the  Digital  Production 
System  (DPS),  it  was  a  transitional  step  which  upgraded 
DMA's  production  system  and  provided  initial  digital 
exploitation  capability.  MARK  85  has  been  delivered  and  is 
in  use  at  DMA  production  centers  (DOD/DMA  DPS  Handbook, 

1989: i'.  There  are  6  segmei  ts  of  the  MARK  85  production 
system.  The  following  are  descriptions  of  these  segments. 

MARK  85  Segments 

DIGITAL  COMPARATOR  (DC/S):  DC/S  provides  the 
capabilities  to  measure  tie  points  and  extract 
elevation  data  from  softcopy  imagery. 

DATA  INTEGRATION  (DI/S):  DI/S  provides  program 
management,  production  management,  source 
collection  and  acquisition  management,  and 
management  of  various  non-imagery  source  libraries 
and  data  bases. 

FEATURE  EXTRACTION  (FE/S):  FE/S  provides  capabilities 
to  perform  feature  extraction  from  hardcopy  imagery  and 
produces  Digital  Feature  Analysis  Data  ( DFAD ) ,  Digital 
Terrain  Analysis  Data  (DTAD) ,  Vertical  Obstruction  Data 
(VOD),  and  elevation  extraction  and  maps/charts  data. 

HARDCOPY  EXPLOITATION  (HE/S):  HE/S  provides  the 
capabilities  to  exploit  hardcopy  imagery  for 
mensuration,  triangulation,  terrain  elevation 
extraction,  point  target  coordinate  derivation, 
rectification,  orthorectif ication,  and  Deployable 
>  Point  Positioning  Data  Base  (PPDB)  cartridge 

generation . 

SOURCE  ACQUISITION  (SA/S):  SA/S  provides  tasking  for 
imagery  collection,  imagery  assessment,  production 
feasibility  analysis,  and  image  library 
management . 
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UNIVERSAL  RECTIFIER  (UR/S):  UR/S  provides  the 
capabilities  to  rectify  softcopy  imagery  and  generate 
PPDB  film  chips. 
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Appendix  C:  HARK  90  SYSTEM 


•« 

General  Information 


MARK  90:  The  second  phase  of  the  Digital  Production 
System  (DPS),  it  provides  an  all-digital  softcopy  production 
system  which  will  improve  production  efficiencies  and 
accommodate  mandatory  conversion  to  exploitation  of  improved 
source  materials.  In  March  1991,  the  MARK  90  system  will 
reach  its  Initial  Operating  Capability  (IOC).  Full 
Operating  Capability  (FOC)  will  occur  in  March  1992  for  all 
31  products  in  the  DPS  baseline  (DOD/DMA  DPS  Handbook, 

1989 :ii).  The  MARK  90  system  will  consist  of  five  MARK  90 
segments  and  the  residual  of  two  MARK  85  segments.  The 
following  is  a  brief  description  of  these  segments. 

MARK  90  Segments 

DATA  EXTRACTION  (DE/S):  DE/S  provides  the  capabilities 
to  extract  terrain  elevation  and  feature  data  from 
softcopy  imagery,  attribute  extracted  features,  edit 
and  merge  new  to  existing  extracted  data,  and  perform 
quality  checks  to  ensure  extracted  data  is  in 
compliance  with  extraction  specifications. 

DATA  SERVICES  (DS/S):  DS/S  is  comprised  of  two 
components,  a  data  base/data  management  subsystem 
and  a  communications  subsystem.  The  data 
base/data  management  subsystem  provides 
capabilities  to  archive  and  provide  access  to  MARK 
90  MC&G  data  and  to  protect  its  integrity.  The 
V  communications  subsystem  of  DS/S  provides  all 

narrowband  communications  for  each  segment's 
internal  communications  for  intersegment 
communications  within  a  center,  and  for 
intercomponent  communications  via  the  Integrated 
*  DMA  Telecommunications  System  (DITS)  within  a 

center.  It  also  provides  wideband  communications 
for  imagery  to  the  production  segments.  The 
communi cat ions  wideband  service  provides  the 
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capabilities  to  read  compressed  softcopy  imagery 
from  Magnetic  Tape  Cassettes  (MTCs)  and  provide  it 
to  the  MARK  90  production  elements  (SP/S,  DE/S, 

PG/S)  within  a  production  center. 

PRODUCT  GENERATION  (PG/S):  PG/S  provides  the  capability 
to  extract  and  attribute  feature  data  from 
monoscopic  imagery  and  cartographic  sources,  to 
extract  air  facilities  and  flight  information 
data,  to  evaluate  and  process  bathymetric  data,  to 
edit  and  merge  existing  to  extracted  data,  to 
compile  MARK  90  data  into  hardcopy  reproducible 
masters  or  digital  products,  and  to  perform 
quality  checks  to  ensure  production  is  in 
compliance  with  product  specifications. 

PRODUCT  MANAGEMENT  (PM/S):  PM/S  provides  a  single 
integrated  production  management  capability  for 
all  of  DMA.  The  segment  provides  the  capability 
to  perform  Agency  level  programming  and  planning; 
to  schedule,  manage  and  monitor  production  center 
resources  and  production  assignments  and  to 
monitor  and  report  production  progress.  The  PM/S 
also  accounts  for  customer  requirements. 

SOURCE  PREPARATION  (SP/S):  SP/S  provides  the 
capabilities  to  review  and  evaluate  multi-media  source 
materials,  measure  softcopy  imagery  to  support 
triangulation,  produce  point  target  products,  and 
generate  Source  Use  Packages  (SUP)  for  use  by 
production. 


MARK  85  Residual  Segments 

HARDCOPY  EXPLOITATION  (HE/S):  HE/S  provides  the 
capabilities  to  exploit  hardcopy  imagery  for 
mensuration,  triangulation,  terrain  elevation 
extraction,  point  target  coordinate  derivation, 
rectification,  orthorectification,  and  Deployable 
Point  Positioning  Data  Base  (PPDB)  cartridge 
generation . 

SOURCE  ACQUISITION  (SA/S):  SA/S  provides  tasking  for 
imagery  collection,  imagery  assessment,  production 
feasibility  analysis,  and  image  library 
management . 


4. 
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Appendix  D:  DMA  Organizational  Structure 


DMA  Headquarters  (DMAHQ)  provides  high-level  policy 
guidance  and  oversight  to  all  components  concerning 
administrative,  research  and  development,  support  and 
maintenance,  and  production.  The  Transition  Management 
Office  (TM)  assists  the  Director  of  DMA  in  formulation  of 
DMA  policies,  long-term  objectives,  implementation 
strategies,  and  plans  to  assure  an  effective  transition, 
both  managerially  and  technically,  to  an  all  digital 
production  system.  Research  and  Engineering  (RE)  serves  as 
the  staff  proponent  for  DMA  and  all  its  functions. 
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DMA  Systems  Center  (DMASC)  is  responsible  for  the 
requirements  definition,  design,  development,  procurement, 

* 

installation,  transition,  and  maintenance  of  the  Digital 
Production  System  as  well  as  all  other  R&D  activities, 
production  line  maintenance,  and  configuration  management. 

DMA  Telecommunications  Services  Center  (DMATSC)  is 
responsible  for  overall  coordination  of  telecommunications 
activities  across  DMA,  operations  and  maintenance  of  the 
Integrated  DMA  Telecommunication  System  (IDTS),  and  security 
accreditation  and  communication  maintenance. 

The  three  DMA  production  centers,  DMA  Reston  Center 
(DMARC) ,  DMA  Aerospace  Center  (DMAAC) ,  and  DMA  Hydrographic 
Topographic  Center  (DMAHTC),  support  the  Digital  Production 
System  development  and  transition  activities  primarily 
through  two  organizations: 

(1)  Research  and  Engineering  Directorate  (RE)  which 
oversees  the  introduction  and  implementation  of  the  new 
technology,  equipment,  systems  and  techniques  into  the 
center’s  production  operations. 

(2)  Production  Programs  Modernization  Division  (PPM) 
which  is  responsible  for  assuring  that  the  production 
elements  have  the  knowledge,  skills,  and  resources  to  move  a 

r 

segment  or  system  into  a  production  center  while  maintaining 
steady  production.  PPM  ensures  that  the  transition  from 
current  production  procedures  to  automated  methods  proceeds  4 

smoothly  and  with  minimal  impact  to  the  center. 

Source:  (DOD/DMA  DPS  Handbook,  1989:59-61) 
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11  May  1990 


Mr.  P.  R.  Gilliam 
Deputy  Director  Mgt/Tech 
M  Defense  Mapping  Agency 

8613  Lee  Highway 
Fairfax,  VA  22031-2137 

Dear  Mr.  Gilliam: 

Thank  you  for  agreeing  to  participate  in  this  Air  Force 
Institute  of  Technology  (AFIT)  Delphi  survey.  The  purpose  of 
this  research  is  to  identify  logistics  requirements  that 
will  improve  operational  support  for  DMA's  automated 
cartographic  or  photogrammetric  systems.  The  research  will 
integrate  the  required  management  principles,  life  cycle 
phases,  and  logistical  support  elements  for  automated 
cartographic  or  photogrammetric  systems. 

By  its  nature,  Delphi  surveys  are  conducted  in  areas 
where  little  or  no  research  data  exists.  The  survey  is 
administered  to  a  select  group  of  "expert"  respondents; 
therefore,  your  response  is  extremely  important.  You  were 
selected  to  participate  in  this  research  effort  because  your 
experience  and  knowledge  qualify  you  as  a  cartographic  or 
photogrammetric  "expert".  Your  opinions  and  comments  will 
be  combined  with  those  of  other  "experts"  to  develop  a  model 
describing  logistical  support  planning  requirements  for 
automated  cartographic  or  photogrammetric  systems. 

To  assist  in  this  research,  please  complete  the  survey 
and  return  it  in  the  self-addressed  envelope  within  one  week 
of  receipt.  As  soon  as  all  the  responses  are  compiled,  a 
second  iteration  of  the  Delphi  survey  will  be  distributed  to 
you. 

Any  comments,  suggestions,  and  ideas  regarding  this 
research  effort  and  the  model  are  welcome.  Please  be 
assured  that  complete  anonymity  will  be  enforced.  If  you 
have  any  questions  about  the  survey  please  have  a  member  of 
your  staff  call  Major  Robert  McCauley  at  (513)  255-4149  (AV 
785-4149)  or  Thomas  R.  Mann  at  (513)  435-1461. 

A  synopsis  of  the  survey  results  will  be  provided  to 
'y  you  upon  completion.  Thank  you  for  your  time  and  sharing 

your  expertise. 


THOMAS  R.  MANN  2  Atgh 

DMASC/EG  1.  Delphi  Survey 

LTFTT  Student  2.  Return  Envelope 

Graduate  Logistics  Management  Program 
Air  Force  Institute  of  Technology  (AFIT) 
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SURVEY  ON  LOGISTICS  SUPPORT  PLANNING  REQUIREMENTS  FOR 
AUTOMATED  CARTOGRAPHIC  OR  PHOTOGRAMMETRIC  SYSTEMS 


The  purpose  of  this  research  is  to  identify  logistics 
planning  requirements  that  will  improve  operational  support 
for  DMA's  automated  cartographic  or  photogrammetric  systems. 
To  assess  logistics  planning  requirements,  this  survey  uses 
the  Integrated  Logistics  Support  (ILS)  concept  as  a  unifying 
structure  for  the  research.  ILS  is  the  disciplined, 
unified,  and  iterative  approach  to  the  management  and 
technical  activities  necessary  to:  (a)  integrate  support 
considerations  into  system  and  equipment  design;  (b)  develop 
support  requirements  that  are  related  consistently  to 
readiness  objectives,  to  design,  and  to  each  other; 

(c)  acquire  the  required  support;  and  (d)  provide  the 
required  support  during  the  operational  phase  at  minimum 
cost . 

It  is  important  that  all  respondents  share  the  same 
common  understanding  of  important  concepts.  Three 
attachments  describing  the  Logistics  Systems  Management 
Matrix  (LSMM) ,  Design  or  Life  Cycle  Phases,  and  ILS  Elements 
are  provided  along  with  this  questionnaire.  Please  review 
these  attachments  before  you  begin,  and  keep  them  beside  you 
as  you  proceed  with  the  survey  for  easy  reference.  When 
instructed,  please  use  the  Attachments  provided  to  clarify 
the  meanings  of  important  concepts.  Please  complete  the 
survey  and  return  it  in  the  self-addressed  envelope  within 
one  week  of  receipt.  As  soon  as  all  the  responses  are 
compiled,  a  second  iteration  of  the  Delphi  survey  will  be 
distributed . 

Some  general  guidelines  for  completing  this  survey  are 
provided  below. 

a.  When  a  question  requires  a  multiple  choice  answer, 
please  circle  the  appropriate  number  on  that  question. 

b.  When  a  question  calls  for  an  answer  along  a  scale, 
please  circle  the  number  which  most  accurately  reflects  your 
judgement  on  that  question  or  statement. 

c.  When  a  question  requires  a  ranking  response,  please 
rank  order  the  alternatives,  using  "1"  for  the  most 
important  item. 

d.  Please  provide  the  rationale  for  your  answers, 
especially  for  those  areas  where  you  feel  strongly.  Add  any 
illustrations,  examples,  or  experiences  you  have  had  that 
will  help  the  other  participants  understand  your  response. 
Feel  free  to  continue  your  comments  on  the  back  of  the 
survey  sheets. 
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e.  Any  ideas  or  recommendations  you  have  for  improving 
logistics  support  for  automated  cartographic  or 
photogrammetric  systems  should  also  be  included. 

f.  Please  use  the  last  page  of  this  survey  to  provide 
additional  comments  you  feel  are  pertinent  to  this  study. 


THANK  YOU  FOR  PARTICIPATING  IN  THIS  SURVEY. 


TOPIC  1:  BACKGROUND  INFORMATION 


The  requested  information  documents  the  participation  of 
"experts"  from  critical  functional  areas  involved  in  the 
administration,  research  and  development,  operational  use, 
and  support/maintenance  of  cartographic  or  photogrammetric 
systems . 


1.  Your  age  is: 

[1]  Less  than  30 

[2]  31  to  40 

[3]  41  to  50 

[4]  51  to  60 

[5]  More  than  60 


2.  Your  highest  educational  level  obtained  is: 


[1]  High  school  graduate  or  GED 

[2]  Some  college  work 

[3]  Associate  degree 

[4]  Bachelors  degree 

[5]  Some  graduate  work 

[6]  Master's  degree 

[7]  Master's  degree  plus  additional  graduate  studies 

[8]  Ph.  D.  or  equivalent 


3.  How  long  have  you  been  associated  with  Automated 
Cartographic  and/or  Photogrammetric  Systems: 

[1]  Less  than  4  years 

[2]  4  years,  but  less  than  8  years 

[3]  8  years,  but  less  than  12  years 

[4]  12  years,  but  less  than  16  years 

[5]  16  years,  but  less  than  20  years 

[6]  20  years  or  more 


4.  What  organization  do  you  work  for  within  the  Defense 
Mapping  Agency? 

[1]  DMA  Headquarters 

[2]  DMA  Systems  Center 

[3]  DMA  Reston  Center 

[4]  DMA  Hydrographic/Topographic  Center 

[5]  DMA  Aerospace  Center 

[6]  Other  _ (please  specify) 
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5.  How  do  you  categorize  your  organization’s  primary  role 
in  the  delivery/use  of  automated  cartographic  or 
photogrammetric  systems? 

[1]  Administrative 

[2]  Research  and  Development 

[3]  Operational  Use  (Production) 

[4]  Support  and/or  Maintenance 

[5]  Other  _ (please  specify) 

6.  How  long  have  you  been  employed  with  DMA? 

[1]  Less  than  4  years 

[2]  4  years,  but  less  than  8  years 

[3]  8  years,  but  less  than  12  years 

[4]  12  years,  but  less  than  16  years 

[5]  16  years,  but  less  than  20  years 

[6]  20  years  or  more 


7.  How  long  have  you  worked  in  your  present  position? 

[1]  Less  than  1  year 

[2]  1  year,  but  less  than  2  years 

[3]  2  years,  but  less  than  3  years 

[4]  3  years,  but  less  than  4  years 

[5]  4  years,  but  less  than  5  years 

[6]  5  years  or  more 


8.  What  other  organizations  within  DMA  have  you  had 
experience  in?  (Check  all  appropriate  choices) 

[1]  DMA  Headquarters 

[2]  DMA  Systems  Center 

[3]  DMA  Reston  Center 

[4]  DMA  Hydrographic/Topographic  Center 

[5]  DMA  Aerospace  Center 

[6]  Other  _ (please  specify) 


9.  Have  you  had  experience  as  a  user  of  DMA  automated 
cartographic  or  photogrammetric  systems  outside  of  DMA? 

[1]  Yes  (If  yes,  answer  question  10) 

[2]  No 


10. 


If  so.  What  organization(s)  outside  of  DMA? 
_  (please  specify) 


With  which  DMA  system(s)  have  you  had  experience? 
_  (please  specify) 


* 
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TOPIC  2:  BASIC  CHARACTERISTICS 


For  questions  numbered  11  through  23,  please  refer  to 
the  three-dimensional  model  known  as  the  Logistics  Systems 
Management  Matrix  (LSMM),  provided  at  Attachment  #1. 
Research  has  suggested  that  a  model  such  as  the  Logistics 
Systems  Management  Matrix  (LSMM)  forms  the  basis  for  proper 
implementation  of  any  given  system.  Certain  sets  of 
questions  will  require  Attachments  #2  and  #3  to  be  used  in 
conjunction  with  Attachment  #1.  Specific  instructions  will 
alert  you  to  these  special  situations.  The  following 
questions  will  assist  in  determining  the  applicability  of 
LSMM  to  automated  cartographic  or  photogrammetric  systems. 


11.  The  LSMM  provides  a  basic  structure  for  outlining  the 
logistics  support  of  automated  cartographic  or 
photogrammetric  systems. 


1 

2 

3 

4 

5 

highly 

disagree 

neither  agree 

agree 

highly 

disagree 

nor  disagree 

agree 

12.  Hpw 

would  you 

change  the  LSMM? 

Please  use  Attachment  #2  in  conjunction  with  Attachment  tl 
to  answer  Questions  #13  through  #16. 

13.  An  understanding  of  design  or  life  cycle  processes  is 
important  for  logistics  support  planning  of  automated 
cartographic  or  photogrammetric  systems. 


1 

2 

3 

4 

5 

highly 

disagree 

disagree 

neither  agree 
nor  disagree 

agree 

highly 

agree 

14.  This  model  accurately  portrays  the  design  or  life  cycle 
phases  of  automated  cartographic  or  photogrammetric  systems 
employed  by  DMA. 


1 

2 

3 

4 

5 

highl y 

disagree 

neither  agree 

agree 

highly 

disagree 

nor  disagree 

agree 

15.  Rank  order  from  1  to  6  the  design  or  life  cycle  phases 
in  terms  of  importance  for  ^ogistics  support  planning  of  DMA 
automated  cartographic  or  photogrammetric  systems?  (Use  "1" 
to  denote  the  most  important). 


Conceptual  Phase 

Advanced  Development  Phase 

Detailed  Design  Phase 

Production 

Operational  Use 

Retirement 


16.  How  would  you  change  the  design  or  life  cycle  phases 
proposed  in  the  LSMM? 


Please  use  Attachment  #3  in  conjunction  with  Attachment  tl 
to  answer  Questions  #17  through  t23. 


17.  Understanding  Integrated  Logistics  Support  (ILS)  is 
important  for  logistics  support  planning  of  automated 
cartographic  or  photogrammetric  systems. 


1 

2 

3 

4 

5 

*■ 

highly 

disagree 

disagree 

neither  agree 
nor  disagree 

agree 

highly 

agree 
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18.  This  model  accurately  portrays  the  ILS  elements  app’ied 
to  automated  cartographic  or  photogrammetric  systems 
employed  by  DMA. 


1 

2 

3 

4 

5 

highly 

disagree 

neither  agree 

agree 

highly 

disagree 

nor  disagree 

agree 

19.  For  the  ILS  elements  listed  below,  what  are  the  three 
most  important  and  the  three  least  important  ILS  elements 
associated  with  logistics  support  planning  of  automated 
cartographic  or  photogrammetric  systems. 

Using  the  left  column  rank  the  three  most  important  ILS 
elements  (Use  "1”  to  denote  the  first,  ,,2,,  to  denote  the 
second,  "3"  to  denote  the  third).  Using  the  right  column 
rank  the  three  least  important  ILS  elements  (Use  "8"  to 
denote  the  eighth,  "9"  to  denote  the  ninth,  "10"  to  denote 
the  tenth) . 


Most  Important  Elements 

_  Maintenance  Planning 

_  Manpower/Personnel 

_  Supply  Support 

_  Support  Equipment 

_  Technical  Data 

_  Training 

_  Computer  Resources 
Support 

_  Facilities 

_  Packaging,  Handling  & 
Transportation 

_  Design  Interface 


Least  Important  Elements 

_  Maintenance  Planning 

_  Manpower/Personnel 

_  Supply  Support 

_  Support  Equipment 

_  Technical  Data 

_  Training 

_  Computer  Resources 

Support 

_  Facilities 

_  Packaging,  Handling  & 

Transportation 

_  Design  Interface 
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20.  Which  of  the  ILS  elements  are  not  applicable  to 
automated  cartographic  or  photogrammetric  systems?  Why  are 
they  not  applicable? 


21.  How  would  you  change  any  of  the  ILS  elements  proposed 
in  the  LSMM? 


22.  Are  there  any  elements  in  the  logistics  support 
planning  of  automated  cartographic  or  photogrammetric 
systems  not  addressed  by  the  LSMM  model?  Please  specify  and 
describe  what  these  elements  are. 


23.  To  what  extent  does  DMA  employ  ILS  in  logistics  support 
requirements  planning  of  automated  cartographic  or 
photogrammetric  systems? 


1 

Not  At 
All 


3 

To  A 
Moderate 
Extent 


5 

To  A 

Very  Great 
Extent 


2 


4 


TOPIC  3:  ILS  ELEMENTS  AND  LIFE  CYCLE  DETERMINATION  &  DESIGN 
REQUIREMENTS 

For  questions  numbered  24  through  43  please  reference 
Attachment  #2  and  #3.  Life  cycle  phases  are  defined  in 
Attachment  #2  and  ILS  elements  are  defined  in  Attachment  #3 
to  provide  assistance  in  answering  questions  24  through  43. 

Provide  comments  on  what  factors,  within  each  of  the  ILS 
elements  listed  below,  are  unique  and  require  logistics 
support  planning  for  automated  cartographic  or 
photogrammetric  systems.  After  providing  comments  on  each 
ILS  element,  please  complete  the  questions  concerning  when 
in  the  life  cycle  of  automated- cartographic  or 
photogrammetric  systems  should  determination  &  design 
requirements  for  each  ILS  element  take  place. 


24.  Maintenance  Planning:  Please  identify  and  explain 
maintenance  planning  factors  unique  to  automated 
cartographic  or  photogrammetric  systems.  Provide  examples 
of  past  experiences  to  support  your  comments. 


25.  At  what  stage  of  the  automated  systems'  life  cycle 
should  maintenance  planning  requirements  determination  and 
design  take  place? 


1 

Conceptual 

Phase 


2 

Advanced 

Development 

Phase 


3 

Detailed 

Design 

Phase 


4 

Production 

Phase 


5 

Operational 

Use 


26.  Manpower  and  Personnel :  Please  identify  and  explain 
manpower  and  personnel  factors  unique  to  automated 
cartographic  or  photogrammetric  systems.  Provide  examples 
of  past  experiences  to  support  your  comments. 


27.  At  what  stage  of  the  automated  systems'  life  cycle 
should  manpower  and  personnel  requirements  determination  and 


design  take 

place? 

1 

2 

3 

4 

5 

* 

Conceptual 

Advanced 

Detailed 

Production 

Operational 

Phase 

Devel opment 

Design 

Phase 

Use 

Phase 

Phase 

4. 

28.  Supply 

Support:  Please  identify  and  explain  supply 

support  factors  unique  to 

automated  cartographic  or 

photogrammetric  systems. 

Provide 

examples  of 

past 

experiences  to  support  your  comments. 


29.  At  what  stage  of  the  automated  systems'  life  cycle 
should  supply  support  requirements  determination  and  design 
take  place? 


1 

Conceptual 

Phase 


2 

Advanced 

Development 

Phase 


3 

Detailed 

Design 

Phase 


4 

Production 

Phase 


5 

Operational 

Use 


30.  Support  Equipment:  Please  identify  and  explain  support 
equipment  factors  unique  to  automated  cartographic  or 
photogrammetric  systems.  Provide  examples  of  past 
experiences  to  support  your  comments. 


31.  At  what  stage  of  the  automated  systems'  life  cycle 
should  support  equipment  requirements  determination  and 
design  take  place? 
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Conceptual 

Phase 
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Detailed 
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Phase 

Design 

Phase 

Phase 

Use 
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32.  Technical  Data:  Please  identify  and  explain  technical 
data  factors  unique  to  automated  cartographic  or 
photogrammetric  systems.  Provide  examples  of  past 
experiences  to  support  your  comments. 


♦ 


33.  At  what  stage  of  the  automated  systems’  life  cycle 
should  technical  data  requirements  determination  and  design 
take  place? 
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Development 

Phase 

Design 

Phase 

Phase 

Use 

34.  Training:  Please  identify  and  explain  training  factors 
unique  to  automated  cartographic  or  photogrammetric  systems. 
Provide  examples  of  past  experiences  to  support  your 
comments . 


35.  At  what  stage  of  the  automated  systems’  life  cycle 
should  training  requirements  determination  and  design  take 
place? 
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Phase 

Design 

Phase 

Phase 
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36.  Computer  Resources  Support:  Please  identify  and  explain 
computer  resources  support  factors  unique  to  automated 
cartographic  or  photogrammetric  systems.  Provide  examples 
of  past  experiences  to  support  your  comments. 


37.  At  what  stage  of  the  automated  systems’  life  cycle 
should  computer  resources  support  requirements  determination 


and  design 

take  place? 
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Phase 

Design 

Phase 

Phase 

Use 

38.  Facilities :  Please  identify  and  explain  facilities 
factors  unique  to  automated  cartographic  or  photogrammetric 
systems.  Provide  examples  of  past  experiences  to  support 
your  comments. 


39.  At  what  stage  of  the  automated  systems'  life  cycle 
should  facilities  requirements  determination  and  design  take 
place? 
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Conceptual 

Phase 
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Advanced 

Development 

Phase 
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Detailed  Production 
Design  Phase 

Phase 
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40.  Packaging.  Handling  &  Transportation:  Please  identify 
and  explain  packaging,  handling,  &  transportation  factors 
unique  to  automated  cartographic  or  photogrammetric  systems. 
Provide  examples  of  past  experiences  to  support  your 
comments . 


* 


41.  At  what  stage  of  the  automated  systems'  life  cycle 
should  packaging,  handling  &  transportation  requirements 
determination  and  design  take  place? 

1  2  3  4  5 

Conceptual  Advanced  Detailed  Production  Operational 

Phase  Development  Design  Phase  Use 

Phase  Phase 


42.  Design  Interface:  Please  identify  and  explain  design 
interface  factors  unique  to  automated  cartographic  or 
photogrammetric  systems.  Provide  examples  of  past 
experiences  to  support  your  comments. 


43.  At  what  stage  of  the  automated  systems'  life  cycle 
should  design  interface  requirements  determination  and 
design  take  place? 
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Production 

Phase 
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ADDITIONAL  COMMENTS: 

How  well  does  DMA  apply  ILS  or  any  other  logistics 
support  planning  process  to  automated  cartographic  or 
photogrammetric  systems?  Please  describe  any  improvements 
which  need  to  be  made  to  this  process.  How  would  you 
suggest  these  improvements  be  accomplished? 
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LOGISTICS  SYSTEMS  MANAGEMENT  MATRIX  (LSMM) 


Research  has  suggested  that  a  model  such  as  the 
Logistics  Systems  Management  Matrix  (LSMM)  below  forms  the 
basis  for  proper  implementation  of  any  given  system.  The 
LSMM  is  a  three-dimensional  model  with  principles  of 
management  on  the  "X"  axis,  design  or  life  cycle  processes 
on  the  "Y"  axis,  and  the  ILS  elements  on  the  "Z"  axis. 
Please  use  Attachment  tl  as  a  reference  in  answering 
questions  11  through  22. 
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The  Logistics  Systems  Management  Matrix 
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DESIGN  OR  LIFE  CYCLE  PHASES 


1.  Conceptual  Phase:  This  phase  is  where  the  product  or 
service  system  is  defined,  acquisition  requirements  are 
determined,  production  and  operational  support  requirements 
are  identified  and  logistics  support  planning  is 
established. 

2.  Advanced  Development  Phase:  This  phase  identifies  and 
defines  the  product  configuration.  It  deals  with  product 
objectives,  performance  limits,  areas  of  risk,  alterations 
and  acquisition  methodologies.  During  this  phase,  logistics 
support  analysis  is  conducted  to  verify  design  concepts, 
followed  by  the  initiation  of  the  logistics  support  plan. 

3.  Detailed  Design  Phase:  This  phase  includes  the  design 
and  test  of  the  product.  It  consists  of  design  effects  on 
reliability,  maintainability,  sustainability,  human  factors 
issues  and  logistics  support.  Engineering  modes  and 
prototypes  are  developed.  At  this  time,  the  formal  logistic 
support  plan  is  completed. 

4.  Production  Phase:  This  phase  consists  of  producing  the 
product  and  its  support,  such  as  test  and  support  equipment, 
spare  parts,  training  personnel,  software,  physical 
distribution  and  transportation,  and  warehousing. 

5.  Operational  Use:  This  phase  covers  all  of  the 
activities  associated  with  consumer  use  and  logistics 
support,  to  include  maintenance  and  sustainability. 

6.  Retirement :  This  phase  consists  of  all  of  the  actions 
necessary  to  phase  out  and  dispose  of  the  system,  to  include 
waste  management,  recycling  or  other  responsive  actions. 

Fast  obsolescence  systems,  such  as  automated  cartographic  or 
photogrammetric  systems,  require  retirement  planning  early 
in  the  life  cycle  to  ensure  proper  phase-out  of  old  systems 
and  implementation  of  new  systems. 
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ILS  ELEMENTS 


The  following  ILS  definitions  come  from  the  Integrated 
Logistics  Support  Guide  published  by  the  Defense  Systems 
Management  College.  Each  definition  is  a  short  synopsis 
for  that  particular  ILS  element.  Please  use  the  ILS 
definitions  provided  in  Attachment  #2  as  a  reference  in 
answering  Questions  23  through  42. 

1.  Maintenance  Planning:  The  process  conducted  by  the 
Government  and  contractor  to  explore  alternatives  and  to 
develop  the  maintenance  concepts  and  maintenance 
requirements  for  the  life  of  the  material  system. 

Maintenance  planning  is  the  lead  analytical  activity  and 
provides  input  to  the  development  of  all  of  the  remaining 
logistics  support  elements. 

2.  Manpower  and  Personnel :  The  element  which  encompasses 
the  identification  and  acquisition  of  military  and  civilian 
personnel  with  the  skills  and  grades  required  to  operate  and 
support  a  material  system  over  its  lifetime  at  peacetime  and 
wartime  rates. 

3.  Supply  Support:  The  element  which  encompasses  all 
actions  required  to  identify  and  obtain  the  spares  and 
repair  parts  needed  to  support  peacetime  and  wartime 
readiness  objectives. 

4.  Support  Eguipment:  The  element  which  encompasses  all 
equipment  required  to  support  operation  and  maintenance  of 
the  material  system.  This  includes  ground  handling 
equipment,  tools,  metrology  and  calibration  equipment,  test 
equipment,  and  logistic  support  for  the  support  equipment. 

5.  Technical  Data:  The  element  which  encompasses  all 
recorded  information  of  a  scientific  or  technical  nature 
related  to  a  program.  Technical  data  are  written 
instructions  such  as  drawings;  operating  and  maintenance 
manuals;  specifications,  inspection,  test  and  calibration 
procedures;  and  computer  programs  which  guide  personnel 
performing  operations  and  support  tasks. 

6.  Training :  The  element  which  encompasses  all  of  the 
processes,  procedures,  techniques,  training  devices  and 
equipment  used  to  train  personnel  to  operate  and  support  a 
material  system.  Examples  include  individual  and  group 
training;  new  equipment  training;  initial,  formal  and  on- 
the-job  training;  and  logistic  support  planning  for  training 
equipment . 
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7.  Computer  Resources  Support:  The  element  is  defined  as 
all  computer  equipment,  software,  associated  documentation, 
contractual  services,  personnel,  and  supplies  needed  to 
operate  and  support  an  embedded  computer  system. 

8.  Facilities :  The  element  which  encompasses  those  real 
property  assets  required  to  support  the  material  system,  and 
the  studies  which  define  types  of  facilities  or  facility 
improvements,  locations,  space  needs,  etc. 

9.  Packaging,  Handling  &  Transportation:  The  element  which 
includes  the  characteristics,  action  and  requirements 
necessary  to  ensure  the  capability  to  transport,  preserve, 
package,  and  handle  all  equipment  and  support  items. 

10.  Design  Interface:  The  element  which  is  an  interactive 
relationship  of  logistics-related  design  parameters,  such  as 
R&M,  to  readiness  and  logistic  support  resource 
requirements . 
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Appendix  F :  Round  One  Delphi  Survey  Comments 


Basic  Characteristics  of  the  LSMM  Model . 

Basic  Structure . 

How  would  you  change  the  LSMM? 

"Reverse  the  Z-Axis  so  that  maintenance  planning  is  at  the 
same  origin  point  as  planning  and  conceptual  phase,  this 
will  show  the  growth  of  a  3-D  solid  from  ground  up." 

"Add  resources  =  $$." 

"Add  a  new  element  on  the  "Y"  axis  between  Production  and 
Operational  Use  entitled  "Transition"  to  allow  an 
implementation  period  for  the  new  system." 

"Restructure  it  to  apply  to  DMA;  training,  facilities, 
maintenance  and  production  are  all  handled  by  different 
directorates  within  DMA  rather  than  under  one  management 
structure. " 

"Include  test  equipment  as  a  separate  element." 

"Software  maintenance  and  continual  enhancement  is  perhaps 
the  most  critical  item  in  the  continuing  life  cycle  support 
of  automated  cartographic  and  photogrammetric  systems.  It  is 
inadequately  addressed  by  lumping  software  under  Technical 
Data  and  Computer  Resources  Support." 

"Add  delegation  to  the  "X"  axis.  Delete  "advanced"  on  the 
"Y"  axis  and  add  delivery,  installation,  test  and 
acceptance,  and  operational  refinement.  Add  maintenance  of 
system  to  the  "Z"  axis." 

"Explain  the  "Z"  axis  relationships.  Are  they  rank  ordered? 

I  am  assuming  they  are  a  listing  used  to  determine  what  ILS 
elements  are  required  during  different  management  and  life 
cycle  phases. 

"I  believe  reviewing  is  part  of  the  "X"  axis." 

"Don't  have  enough  experience  or  knowledge  with  the  model  to 
suggest  changes-  Seems  comprehensive. ' 

"All  the  blocks  are  the  same  size.  If  the  amount  of 
resources  could  be  incorporated,  the  blocks  would  be 
variably  sized.  Also,  "Advanced  Design  Phase"  should  be 
"Preliminary  Design  Phase".  For  most  development  today,  the 


terminology  is  (1)  Design  Concept;  (2)  Preliminary  Design; 

(3)  Critical  Design.  Remove  "Planning"  from  "Maintenance 
Planning".  Is  this  a  solid  volume?  Or  are  there  holes  in  the 
middle  if  some  of  the  blocks  really  don't  apply  (such  as 
Computer  Resources,  Organizing  for  Retirement)?" 

"Change  the  description  of  the  LSMM  on  attachment  #1  to 
read:  . . . ,  forms  the  basis  for  proper  implementation  of  a_ 
logistics  support  system  for  any  given  system.  If  it  is  * 

intended  to  describe  the  implementation  of  a  total  system 
end-to-end,  it  is  too  weak  in  the  front-end  dealing  with 
requirements  definition."  Add  "Funding"  to  the  "X"  axis." 

"Separate  maintenance  into  hardware  and  software 
maintenance . " 

Design  or  Life  Cycle  Processes . 

How  would  you  change  the  design  or  life  cycle  phases 
proposed  in  the  LSMM? 

"Sof tware(modif ication,  configuration  control,  upgrade,  and 
interface  with  external  system)  should  be  fully  addressed  in 
Detailed  Design  Phase." 

"Add  Transition  Phase  between  Production  and  Operational 
Use.  " 

"Combine  conceptual  and  advanced  development  phase." 

"Change  the  names  to  reflect  what  DMA  calls  them-  while  we 
generally  follow  the  DOD  phases  we  call  them  something 
different . " 

"Add  a  process  improvement  block  after  operational  use.  Most 
DMA  systems  go  through  considerable  process  improvements 
after  FOC," 

"The  Operational  phase  is  conceived  as  merely  being  the  use 
of  the  "product".  In  reality,  life  cycle  phases  1-4  normally 
continue  to  reoccur  repeatedly  throughout  the  operational 
life  of  automated  cartographic  and  photogrammetric  systems. 

In  my  experience  seldom  if  ever  has  a  production  system  been 

the  same  "product"  at  the  end  of  its  life  cycle  as  was 

envisioned  at  the  start  of  its  operational  use."  *■ 

"There  is  too  much  variation  in  "Operational  Use"  to  lump  it 
all  together.  Break  it  out  into  three  categories:  initial 
burn  in  period,  steady  state,  and  advanced  age." 

"We  should  just  stick  to  them.  Now  we  almost  always  take  an 
R  &  D  prototype  system  and  make  it  a  production  system." 
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"Change  arrangement  to  Design  Concept,  Preliminary  Design, 
Critical  Design,  Build-to,  Production,  and  Retirement." 

"For  developing  logistics  support,  the  phases  are  accurate 
as  described.  ' 

"They  should  agree  with  the  new  DOD  5000.1." 

Integrated  Logistics  Support  ( ILS)  Elements . 

Which  of  the  ILS  elements  are  not  applicable  to 
automated  cartographic  or  photogrammetric  systems?  Why 
are  they  not  applicable? 

"Design  Interface-  I  am  not  sure  what  this  is  supposed  to  be 
with  respect  to  these  systems." 

"The  Design  Interface  is  often  not  applicable.  Much  of  the 
hardware  is  Commercial  Off-the-shelf  (COTS)  and  we  have 
little  input  into  its  design." 

"Manpower/Personnel-  DMA  is  authorized  "X"  numbers  of 
employees  yearly.  The  priority  and  production  rates 
determine  the  numbers  of  people  to  operate  these  systems." 

"All  are  applicable,  all  are  important,  but  some  can  be  made 
part  of  the  Acquisition  Contract." 

"Manpower/Personnel-  not  part  of  ILS  part  of  programming  and 
budgeting  process." 

"Manpower/Personnel  may  not  be  applicable  in  the  DMA 
environment  because  of  our  structured  environment.  The  time 
frames  involved  in  justifying  and  staffing  new  positions, 
particularly  in  a  time  of  declining  budgets  makes  it 
difficult  to  get  the  people  needed  in  the  time  frame 
involved. " 

"If  anyone  says  an  ILS  element  is  not  applicable  to 
automated  cartographic  or  photogrammetric  systems  I  would 
discount  his/her  entire  survey-  the  individual  obviously 
doesn't  comprehend  ILS." 

"We  traditionally  have  not  included  some  in  ILS  planning 
(manpower,  training,  and  computer  resources  support)  but  had 
these  covered  by  people  dealing  with  transition." 

"Packaging,  handling,  and  transportation-  the  DMA  systems 
are  static  and  arc  rarely  moved  once  installed." 

"The  description  of  "Design  Interface"  is  not  clear." 
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How  would  you  change  any  of  the  ILS  elements  proposed 
in  the  LSMM? 

"Design  interface  needs  to  be  more  clearly  described  and 
explained. " 

"There  should  be  a  separate  element  for  applications 
software.  Software  should  not  be  lumped  under  Technical 
data." 

"Would  recognize  that  development  contractor  should/could 
provide  some  of  the  elements." 

"I  don't  think  design  interface  is  applicable." 

"Add  Security  Support." 

"For  many  of  the  elements  the  support  required  for  the 
development  is  determined  separately  from  that  for 
maintenance.  These  include:  Manpower,  Training,  Computer 
Resources  and  Facilities.  Maintenance  requirements  in  these 
areas  are  only  part  of  the  total  requirement." 

Are  there  any  elements  in  the  logistics  support 
planning  of  automated  cartographic  or  photogrammetric 
systems  not  addressed  by  the  LSMM  model?  Please 
specify  and  describe  what  these  elements  are. 

"Contractor  interface  important  where  equipments  are  very 
unique i " 

"Highly  dependent  on  communication  -  narrow  band  and  wide 
band,  fiber  optics,  ethernet,  and  switching  and  transmission 
gear;  may  need  a  separate  communication  category." 

"ILS  documentation  usually  contains  a  safety  chapter  or 
annex.  Safety  needs  to  be  addressed  in  your  model,  but  I  am 
not  sure  how  because  it  cuts  across  most  of  your 
categories . " 

"This  model  fails  to  recognize  the  fundamental  nature  of 
these  systems,  which  is  that  they  are  a  part  of  a  production 
process .  These  processes  and  the  systems  necessarily  change 
and  evolve  as  improvements  are  made  to  increase  productivity 
and  as  new  or  changed  products  are  introduced  into  the 
production  line.  Automated  cartographic  or  photogrammetric 
systems  evolve  during  their  life  cycle  so  that  they  are 
often  very  different  from  their  initial  configuration  and 
use  in  operation.  Changes  are  made  in  both  system  hardware 
and  software  to  effect  this  evolution.  The  ability  to 
constantly  modify  and  improve  applications  software  is 
particularly  crucial  to  the  production  use  of  these  systems. 
Applications  software  is  very  much  an  extension  of  the 
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cartographer  or  photogrammetrist .  It  is  probably  not  useful 
or  meaningful  to  try  to  separate  conceptually  software 
maintenance  from  software  development  or  enhancement.  The 
basic  problem  with  the  model  presented  in  that  it  is  linear, 
when  the  actual  life  cycle  for  these  systems  involves  an 
iterative  development  and  operational  cycle.” 

"Should  security  be  added  as  an  additional  logistics 
element?" 

"If  you  include  DMA's  traditional  "transition"  events  then 
you  are  missing  Demonstrations  (to  prove  system  meets 
requirements)  and  Exercise  and  Rehearsals  (to  familiarize 
workers  with  the  system  prior  to  production." 

"Security  is  a  major  issue  and  requires  special  attention. 
Calibration  is  a  significant  element  on  some  systems." 

Logistics  Support  Planning  Applications  within  DMA. 

How  well  does  DMA  apply  ILS  or  any  other  logistics 
support  planning  process  to  automated  cartographic  or 
photogrammetric  systems?  Please  describe  any 
improvements  which  need  to  be  made  to  this  process. 

How  would  you  suggest  these  improvements  be 
accomplished? 

"Currently,  the  planning  process  is  not  done  as  soon  as  it 
should.be-  on  small  procurements  it  is  an  afterthought. 
Personnel  need  to  be  trained  to  think  ILS  during 
development . " 

"Very  poorly.  There  is  no  integrated  approach  to  logistics 
support  planning.  Individual  elements  are  addressed 
separately  and  some  of  those  elements  are  not  even 
considered  part  of  the  ILS  process.  The  only  real  elements 
DMA  addresses  as  part  of  logistics  support  are  maintenance 
supply,  and  support  and  test  equipment.  The  other  elements 
are  not  really  recognized  as  part  of  the  ILS  process.  DMA 
needs  to  place  more  emphasis  on  integrating  all  of  the 
elements  into  a  comprehensive  and  cohesive  ILS  program.  More 
emphasis  needs  to  placed  on  design  interface  in  order  to 
influence  design  for  supportability  in  the  early  phases  of 
system  development.  Additionally,  the  program  managers  must 
be  made  responsible  for  considering  life  cycle  support,  and 
therefore  ILS,  during  system  design  and  manufacture.  The 
PM's  must  recognize  their  responsibility  for  integrating  ILS 
as  one  of  the  program  management  elements." 

"DMA  needs  more  involvement  during  the  Advanced  Development 
and  Detailed  Design  Phases  in  order  to  do  a  better  job  of 
supporting  cartographic  and  photogrammetric  systems. 
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"From  my  perspective,  DMA  does  not  do  a  good  job  of 
planning . " 

"Poorly-  we  are  at  least  one  year  behind  but  we  have 
contractors  in  place  to  support  until  we  have  our  sea  legs." 

"Generally,  in  the  past  the  logistics  support  for  production 
systems  has  been  adequate  for  production  needs,  which  would 
indicate  some  planning  process.  Production  departments 
generally  supplied  their  own  software  maintenance. 
Reorganizations  of  maintenance  responsibilities  and 
increased  reliance  of  contractor  maintenance  in  recent  years 
has  generally  lead  to  a  decrease  in  the  level  of 
satisfaction  on  the  production  organizations  in  part." 

"I  saw  little  evidence  of  an  ILS  approach  to  our  past 
systems.  I  do  see  an  attempt  to  provide  ILS  to  our  new 
systems.  Unfortunately,  obstacles  are  constantly  being 
placed  in  our  Why.  Logistics  is  viewed  as  obstacle  by  many 
of  the  COTR  type  people.  We  need  more  influence  earlier  in 
the  system  design  phases." 

"Haphazardly-  not  enough  firm  financial  commitment  to 
adequately  support  a  system.  There  needs  to  be  a  Logistics 
Directorate  whose  sole  responsibility  is  maintenance  without 
interference  from  the  Centers  or  other  departments  with  the 
exception  of  communication  during  times  of  crisis.  DMA  needs 
to  get  a  vision  of  maintenance  as  supporting  production 
rather  than  "fix  it  if  it  breaks".  I  would  call  a  strong 
logistics  organization  in  DMA  a  safeguard  on  the  vast 
investment  the  government  has  made  in  DMA." 

"DMA  appears  to  do  a  great  deal  of  planning  but  does  not  do 
a  good  job  of  implementing  the  plan.  Maintenance  people  are 
often  not  adequately  trained  to  support  the  system,  spare 
parts  are  often  not  maintained,  equipment  status  reports  are 
not  adequately  reviewed  or  trend  analysis  performed  and 
preventive  maintenance  is  often  not  performed.  I  would 
review  the  completed  DMA  support  function  from  top  to  bottom 
and  execute  a  plan  to  raise  the  level  of  performance. 

"Very  poorly-  Didn’t  apply  the  ILS  early  enough.  Minimal 
planning  in  early  stages.  Now  we  are  playing  catch  up 
through  the  RFC  process." 

"Fair-  More  responsiveness/accountable  to  Production  Center 
requirements  with  Cost  Benefit  Analysis  of  what  work/$  = 
most  beneficial  to  DMA,  not  who  has  what  training  and  is 
available  or  what  crisis  happens  to  call  for  $  when  $  are 
avai 1 abl e . " 

"Varies-  MARK  85  and  MARK  90  ILS  very  well  done.  Piecemeal 
buys  are  of  great  variety,  some  good  and  some  non-existent." 
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"The  one  area  that  DMA  historically  has  ignored  is  the 
application  of  ILS  during  the  conceptual  phase  of 
development  rather  than  at  the  operational  phase.  MARK  85 
and  MARK  90  systems  are  examples  of  how  DMA  is  changing  with 
respect  to  ILS.  DMA  needs  to  follow  the  formal  processes  of 
the  life  cycles  of  systems  in  a  more  rigorous  manner  for  all 
systems,  as  is  being  done  for  MARK  90." 

"I  believe  that  DMA  does  a  reasonably  good  job  of  dealing 
with  systems  development  and  operation.  However,  we  do  not 
approach  it  in  the  same  way  as  espoused  by  the  USAF.  In 
recent  times  we  have  had  agency  directors  who  have  tried  to 
dictate  organizational  and  process  changes  to  get  more  in 
line  and  I  have  yet  to  see  the  benefits-  perhaps  somewhere 
in  the  future  they  will  pay  off." 

"DMA  hasn't  done  this  very  well  in  the  past.  As  a  result, 

DMA  Director  is  creating  a  Directorate  for  Acquisition, 
Installation,  and  Logistics.  ADD/Logistics  includes  four 
divisions:  Logistics  Plans,  Distribution,  Maintenance,  and 
Supply.  This  will  eliminate  many  of  DMA’s  fragmented 
logistics  activities  and  result  in  a  proactive  consolidated 
logistics  organization  that  is  more  capable  of  developing, 
planning,  and  implementing  true  ILS." 

"We  take  too  long  to  start  the  process,  as  a  result  ILS 
doesn't  do  what  it  is  intended  to  do  (decisions  have  already 
been  made)." 

"We  have  done  reasonably  well  with  the  Digital  Production 
System  and  subsequent  large  procurements." 

"ILS  is  a  major  player.  Systems  could  not  be  delivered  or 
maintained,  facilities  would  not  be  adequate,  and  personnel 
wouldn't  be  trained  without  ILS.  ILS  has  always  been  a  fuzzy 
area  that  I  always  relied  on  a  logistics  person  to  work.  I 
provided  the  logistics  office  the  technical  specs  of  what 
was  needed  by  when,  and  they  always  came  through." 

"Not  very  well-  The  MARK  90  development  is  trying  to 
establish  system  level  requirements  documents  for  ILS  which 
will  help.  Problem  is  that  content  requirements  of  the 
documents  are  overlooked.  Biggest  improvement:  Start 
planning  earlier." 


Appendix  G:  Round  Two  Delphi.  Survey 

20  June  1990 

Mr.  M.  Z.  Labovitz 
Director,  Acquisition  Office 
Defense  Mapping  Agency 
8613  Lee  Highway 
Fairfax,  VA  22031-2137 

Dear  Mr.  Labovitz: 

Thank  you  for  completing  the  first  round  of  the  AFIT 
Delphi  Survey  on  Logistics  Support  Planning  Requirements  for 
Automated  Cartographic  or  Photogrammetric  Systems.  Your 
comments  were  of  great  value  to  this  research.  Enclosed  you 
will  find  the  second  round  of  the  Delphi  survey  containing 
respondent  feedback.  Please  return  your  completed  second 
round  survey  within  one  week:  it  is  essential  to  have  timely 
and  complete  participation  in  order  to  assure  successful 
completion  of  the  Thesis  effort. 

I  am  certain  you  will  find  the  comments  of  other  DMA 
experts  interesting.  Not  all  questions  require  answers  in 
the  second  round  because  consensus  was  reached  in  the  first 
round  of  the  Delphi  Survey.  The  criterion  for  consensus  was 
established  at  60%  agreement  on  a  single  response  item.  The 
percent  of  consensus  agreement  is  provided  for  each  answer. 
The  questions  requiring  a  new  response  are  asterisked  and 
are  accompanied  by  the  statement  "Your  New  Response".  Mean 
ratings  and/or  comments  are  provided  to  assist  you  in 
completing  these  questions.  Please  consider  all  the 
feedback  in  making  your  responses  on  the  second  round  of  the 
survey . 

You  should  find  the  second  round  of  the  survey  easier 
and  faster  to  complete,  as  it  requires  fewer  numerical 
responses.  Please  review  the  qualitative  comments  and 
provide  your  reactions  to  and/or  synthesis  of  these 
comments.  If  you  have  any  questions  about  the  survey  please 
call  Major  Robert  McCauley  at  (513)  255-4149  (AV  785-4149) 
or  Thomas  R.  Mann  at  (513)  435-1461. 

A  synopsis  of  the  survey  results  will  be  provided  to 
you  upon  completion.  Thank  you  again  for  taking  the  time 
and  helping  us  learn  more  about  logistics  support  planning 
requirements  for  automated  cartographic  or  photogrammetric 
systems.  Please  return  the  survey  within  one  week. 


THOMAS  R.  MANN  2  Atch 

DMASC/EG  1.  Delphi  Survey 

LTFTT  Student  2.  Return  Envelope 

Graduate  Logistics  Management  Program 
Air  Force  Institute  of  Technology  (AFIT) 
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SURVEY  ON  LOGISTICS  SUPPORT  PLANNING  REQUIREMENTS  FOR 
AUTOMATED  CARTOGRAPHIC  OR  PHOTOGRAMMETRIC  SYSTEMS 


The  purpose  of  this  research  is  to  identify  logistics 
planning  requirements  that  will  improve  operational  support 
for  DMA's  automated  cartographic  or  photogrammetric  systems. 
To  assess  logistics  planning  requirements,  this  survey  uses 
the  Integrated  Logistics  Support  (ILS)  concept  as  a  unifying 
structure  for  the  research.  ILS  is  the  disciplined, 
unified,  and  iterative  approach  to  the  management  and 
technical  activities  necessary  to:  (a)  integrate  support 
considerations  into  system  and  equipment  design;  (b)  develop 
support  requirements  that  are  related  consistently  to 
readiness  objectives,  to  design,  and  to  each  other; 

(c)  acquire  the  required  support;  and  (d)  provide  the 
required  support  during  the  operational  phase  at  minimum 
cost . 

It  is  important  that  all  respondents  share  the  same 
common  understanding  of  important  concepts.  Three 
attachments  describing  the  Logistics  Systems  Management 
Matrix  ( LSMM) ,  Design  or  Life  Cycle  Phases,  and  ILS  Elements 
are  provided  along  with  this  questionnaire.  Please  review 
these  attachments  before  you  begin,  and  keep  them  beside  you 
as  you  proceed  with  the  survey  for  easy  reference.  When 
instructed,  please  use  the  Attachments  provided  to  clarify 
the  meanings  of  important  concepts.  Please  complete  the 
survey  and  return  it  in  the  self-addressed  envelope  within 
one  week  of  receipt.  As  soon  as  all  of  the  second  round 
responses  are  compiled,  a  synopsis  of  the  survey  results 
will  be  distributed. 

Some  general  guidelines  for  completing  this  survey  are 
provided  below. 

a.  Please  consider  the  feedback  provided  before  you 
respond  to  the  questions. 

b.  When  a  question  calls  for  an  answer  along  a  scale, 
please  circle  the  number  which  most  accurately  reflects  your 
judgement  on  that  question  or  statement. 

c.  When  a  question  requires  a  ranking  response,  please 
rank  order  the  alternatives,  using  "1"  for  the  most 
important  item. 

d.  Please  provide  the  rationale  for  your  answers, 
especially  for  those  areas  where  you  feel  strongly.  Add  any 
illustrations,  examples,  or  experiences  you  have  had  that 
will  help  the  other  participants  understand  your  response. 
Feel  free  to  continue  your  comments  on  the  back  of  the 
survey  sheets. 


e.  Any  ideas  or  recommendations  you  have  for  improving 
logistics  support  for  automated  cartographic  or 
photogrammetric  systems  should  also  be  included. 

f.  Please  use  the  last  page  of  this  survey  to  provide 
additional  comments  you  feel  are  pertinent  to  this  study. 

THANK  YOU  FOR  PARTICIPATING  IN  THIS  SURVEY. 
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TOPIC  2:  BASIC  CHARACTERISTICS 


For  questions  numbered  11  through  23,  please  refer  to 
the  three-dimensional  model  known  as  the  Logistics  Systems 
Management  Matrix  (LSHrt),  provided  at  Attachment  #1. 
Research  has  suggested  that  a  model  such  as  the  Logistics 
Systems  Management  Matrix  (LSMM)  forms  the  basis  for  proper 
implementation  of  any  given  system.  Certain  sets  of 
questions  will  require  Attachments  #2  and  #3  to  be  used  in 
conjunction  with  Attachment  #1.  Specific  instructions  will 
alert  you  to  these  special  situations.  The  following 
questions  will  assist  in  determining  the  applicability  of 
LSMM  to  automated  cartographic  or  photogrammetric  systems. 


11.  The  LSMM  provides  a  basic  structure  for  outlining  the 
logistics  support  of  automated  cartographic  or 
photogrammetric  systems. 

ROUND  1  CONSENSUS:  81%  AGREE  OR  HIGHLY  AGREE. 


**12.  How  would  you  change  the  LSMM? 

ROUND  1  COMMENTS: 

"Reverse  the  Z-Axis  so  that  maintenance  planning  is  at  the 
same  origin  point  as  planning  and  conceptual  phase,  this 
will  show  the  growth  of  a  3-D  solid  from  ground  up." 

"Add  resources  =  $$." 

"Add  a  new  element  on  the  "Y"  axis  between  Production  and 
Operational  Use  entitled  "Transition"  to  allow  an 
implementation  period  for  the  new  system." 

"Restructure  it  to  apply  to  DMA;  training,  facilities, 
maintenance  and  production  are  all  handled  by  different 
directorates  within  DMA  rather  than  under  one  management 
structure . " 

"Include  test  equipment  as  a  separate  element." 

"Software  maintenance  and  continual  enhancement  is  perhaps 
the  most  critical  item  in  the  continuing  life  cycle  support 
of  automated  cartographic  and  photogrammetric  systems.  It  is 
inadequately  addressed  by  lumping  software  under  Technical 
Data  and  Computer  Resources  Support." 


"Add  delegation  to  the  "X"  axis.  Delete  "advanced"  on  the 
"Y"  axis  and  add  delivery,  installation,  test  and 
acceptance,  and  operational  refinement.  Add  maintenance  of 
system  to  the  "Z"  axis." 

"Explain  the  "Z"  axis  relationships.  Are  they  rank  ordered? 

I  am  assuming  they  are  a  listing  used  to  determine  what  ILS 
elements  are  required  during  different  management  and  life 
cycle  phases. 

"I  believe  reviewing  is  part  of  the  "X"  axis." 

"Don't  have  enough  experience  or  knowledge  with  the  model  to 
suggest  changes-  Seems  comprehensive." 

"All  the  blocks  are  the  same  size.  If  the  amount  of 
resources  could  be  incorporated,  the  blocks  would  be 
variably  sized.  Also,  "Advanced  Design  Phase"  should  be 
"Preliminary  Design  Phase".  For  most  development  today,  the 
terminology  is  (1)  Design  Concept;  (2)  Preliminary  Design; 
(3)  Critical  Design.  Remove  "Planning"  from  "Maintenance 
Planning".  Is  this  a  solid  volume?  Or  are  there  holes  in  the 
middle  if  some  of  the  blocks  really  don't  apply  (such  as 
Computer  Resources,  Organizing  for  Retirement)?” 

"Change  the  description  of  the  LSMM  on  attachment  #1  to 
read:  ...,  forms  the  basis  for  proper  implementation  of  a 
logistics  support  system  for  any  given  system.  If  it  is 
intended  to  describe  the  implementation  of  a  total  system 
end-to-end,  it  is  too  weak  in  the  front-end  dealing  with 
requirements  definition."  Add  "Funding"  to  the  "X"  axis." 

"Separate  maintenance  into  hardware  and  software 
maintenance . " 


YOUR  NEW  COMMENTS: 


Please  use  Attachment  #2  in  conjunction  with  Attachment  tl 
to  answer  Questions  #13  through  <16. 

13.  An  understanding  of  design  or  life  cycle  processes  is 
important  for  logistics  support  planning  of  automated 
cartographic  or  photogrammetric  systems. 

ROUND  1  CONSENSUS:  100%  AGREE  OR  HIGHLY  AGREE. 


14.  This  model  accurately  portrays  the  design  or  life  cycle 
phases  of  automated  cartographic  or  photogrammetric  systems 
employed  by  DMA. 

ROUND  1  CONSENSUS:  65%  AGREE  OR  HIGHLY  AGREE. 


**15.  Using  the  mean/rank  method,  the  mean  responses  from 
Round  1  were  ranked  from  1  to  6.  The  only  consensus 
response  was  Retirement,  which  was  ranked  #6.  Please  rank 
order  from  1  to  5  the  design  or  life  cycle  phases  in  terms 
of  importance  for  logistics  support  planning  of  DMA 
automated  cartographic  or  photogrammetric  systems?  (Use  Ml" 
to  denote  the  most  important). 

ROUND  1  RESPONSES: 


6 


Detailed  Design  Phase 
Advanced  Development  Phase 
Production  Phase 
Operational  Use  Phase 
Conceptual  Phase 
Retirement 


(ROUND  1  MEAN:  2.387) 
(ROUND  1  MEAN:  2.581) 
(ROUND  1  MEAN:  3.290) 
(ROUND  1  MEAN:  3.323) 
(ROUND  1  MEAN:  3.654) 
(ROUND  1  CONSENSUS:  87%) 


**16.  How  would  you  change  the  design  or  life  cycle  phases 
,  proposed  in  the  LSMM? 

ROUND  1  COMMENTS: 

"Sof tware(modif ication ,  configuration  control,  upgrade,  and 
*  interface  with  external  system)  should  be  fully  addressed  in 

Detailed  Design  Phase." 

"Add  Transition  Phase  between  Production  and  Operational 
Use." 
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"Combine  conceptual  and  advanced  development  phase." 

"Change  the  names  to  reflect  what  DMA  calls  them-  while  we 
generally  follow  the  DOD  phases  we  call  them  something 
different . " 

"Add  a  process  improvement  block  after  operational  use.  Most 
DMA  systems  go  through  considerable  process  improvements 
after  FOC." 

"The  Operational  phase  is  conceived  as  merely  being  the  use 
of  the  "product".  In  reality,  life  cycle  phases  1-4  normally 
continue  to  reoccur  repeatedly  throughout  the  operational 
life  of  automated  cartographic  and  photogrammetric  systems. 
In  my  experience  seldom  if  ever  has  a  production  system  been 
the  same  "product"  at  the  end  of  its  life  cycle  as  was 
envisioned  at  the  start  of  its  operational  use." 

"There  is  too  much  variation  in  "Operational  Use"  to  lump  it 
all  together.  Break  it  out  into  three  categories:  initial 
burn  in  period,  steady  state,  and  advnnced  age." 

"We  should  just  stick  to  them.  Now  we  almost  always  take  an 
R  &  D  prototype  system  and  make  it  into  a  production 
system. " 

"Change  arrangement  to  Design  Concept,  Preliminary  Design, 
Critical  Design,  Build-to,  Production,  and  Retirement." 

"For  developing  logistics  support,  the  phases  are  accurate 
as  described." 

"They  should  agree  with  the  new  DOD  5000.1." 


YOUR  NEW  COMMENTS: 


4 
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Please  use  Attachment  #3  in  conjunction  with  Attachment  tl 
to  answer  Questions  >17  through  t 23. 


17.  Understanding  Integrated  Logistics  Support  (ILS)  is 
important  for  logistics  support  planning  of  automated 
cartographic  or  photogrammetric  systems. 

ROUND  1  CONSENSUS:  94%  AGREE  OR  HIGHLY  AGREE. 


**18.  This  model  accurately  portrays  the  ILS  elements 
applied  to  automated  cartographic  or  photogrammetric  systems 
employed  by  DMA. 

ROUND  1  MEAN:  3.200 

YOUR  NEW  RESPONSE: 


1 

2 

3 

4 

5 

highl y 

disagree 

neither  agree 

agree 

highly 

disagree 

nor  disagree 

agree 

**19.  A  weighting  factor  was  assigned  to  each  response  to 
determine  the  three  HOST  important  ILS  elements  associated 
with  logistics  support  planning  of  automated  cartographic  or 
photogrammetric  systems.  A  value  of  "5"  was  assigned  to 
each  ILS  element  with  a  "1"  response;  a  value  of  "3"  was 
assigned  to  each  ILS  element  with  a  "2"  response;  and  a 
value  of  "1"  was  assigned  to  each  ILS  element  with  a  "3" 
response.  The  total  value  of  each  ILS  element  was  used  to 
determine  the  overall  ranking,  from  TOP  TO  BOTTOM,  of  THE 
MOST  IMPORTANT  ILS  ELEMENTS. 

The  only  consensus  response  was  Maintenance  Planning, 
which  was  found  to  be  the  most  important  or  "1".  Please 
rank  the  second  and  third  most  important  ILS. elements  (Use 
"2"  to  denote  the  second,  "3"  to  denote  the  third). 

Most  Important  Elements 


Maintenance  Planning 

(ROUND 

1 

CONSENSUS:  62%) 

Manpower /Personnel 

(ROUND 

1 

VALUE: 

54) 

Training 

(ROUND 

1 

VALUE : 

35) 

Supply  Support 

(ROUND 

1 

VALUE : 

23) 

Design  Interface 

(ROUND 

1 

VALUE : 

18) 

Technical  Data 

(ROUND 

1 

VALUE : 

16) 

Computer  Resources 
Support 

(ROUND 

1 

VALUE: 

10) 

Facilities 

(ROUND 

1 

VALUE: 

09) 

Support  Equipment 

(ROUND 

1 

VALUE: 

05) 

Packaging,  Handling  & 
Transportation 

(ROUND 

1 

VALUE : 

00) 
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**19A.  A  weighting  factor  w" s  assigned  to  each  response  to 
determine  the  three  LEAST  important  ILS  elements  associated 
with  logistics  support  planning  of  automated  cartographic  or 
photogrammetric  systems.  A  value  of  "5  was  assigned  to 
each  ILS  element  with  a  "10"  response;  a  value  of  "3"  was 
assigned  to  each  ILS  element  with  a  "9"  response;  and  a 
value  of  "1"  was  assigned  to  each  ILS  element  with  a  "8" 
response.  The  total  value  of  each  ILS  element  was  used  to 
determine  the  overall  ranking,  from  TOP  TO  BOTTOM,  of  the 
LEAST  IMPORTANT  ILS  ELEMENTS. 

Please  rank  the  three  least  important  ILS  elements  (Use 
"8"  to  denote  the  eighth,  "9"  to  denote  the  ninth,  "10"  to 
denote  the  tenth) . 

Least  Important  Elements 


Packaging,  Handling  & 
Transportation 

(ROUND 

1 

VALUE: 

110) 

Design  Interface 

( ROUND 

1 

VALUE: 

62) 

Facilities 

(ROUND 

1 

VALUE : 

35) 

Compu.er  Resources 
Support 

(ROUND 

1 

VALUE : 

24) 

Support  Equip  lent 

(ROUND 

1 

VALUF • 

15) 

Supply  Support 

(I'.OUND 

1 

VALUE: 

15) 

Manpower /Personnel 

(POUND 

1 

VALUE : 

09) 

Technical  Data 

(ROUND 

1 

VALUE : 

02) 

Training 

(ROUNr 

1 

VALUE: 

00) 

Maintenance  Planning 

(ROUND 

1 

VALUE: 

00) 

**20.  Which  of  the  ILS  elements  are  not  applicable  to 
automated  cartographic  or  photogrammetric  systems?  Why  are 

♦  they  not  applicable? 

ROUND  1  COMMENTS: 

"Design  Interface-  I  am  not  sure  what  this  is  supposed  to  be 

*  with  respect  to  these  systems." 

"The  Design  Interface  is  often  r.ot  applicable.  M-- -n  of  the 
hardware  is  COTS  and  we  have  little  input  into  its  design." 
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"Manpower/Personnel-  DMA  is  authorized  "X"  numbers  of 
employees  yearly.  The  priority  and  production  rates 
determine  the  numbers  of  people  to  operate  these  systems." 

"All  are  applicable,  all  are  important,  but  some  can  be  made 
part  of  the  Acquisition  Contract." 

"Manpower/Personnel-  not  part  of  ILS  part  of  programming  and 
budgeting  process." 

"Manpower/Personnel  may  not  be  applicable  in  the  DMA 
environment  because  of  our  structured  environment.  The  time 
frames  involved  in  justifying  and  staffing  new  positions, 
particularly  in  a  time  of  declining  budgets  makes  it 
difficult  to  get  the  people  needed  in  the  time  frame 
involved. " 

"If  anyone  says  an  ILS  element  is  not  applicable  to 
automated  cartographic  or  photogrammetric  systems  I  would 
discount  his/her  entire  survey-  the  individual  obviously 
doesn't  comprehend  ILS." 

"We  traditionally  have  not  included  some  in  ILS  planning 
(manpower,  training,  and  computer  resources  support)  but  had 
these  covered  by  people  dealing  with  transition." 

"Packaging,  handling,  and  transportation-  the  DMA  systems 
are  static  and  are  rarely  moved  once  installed." 

"The  description  of  "Design  Interface"  is  not  clear." 

YOUR  NEW  COMMENTS: 


**21.  How  would  you  change  any  of  the  ILS  elements  proposed 
in  the  LSMM? 

ROUND  1  COMMENTS: 

"Design  interface  needs  to  be  more  clearly  described  and 
explained. " 

"There  should  be  a  separate  element  for  applications 
software.  Software  should  not  be  lumped  under  Technical 
data." 

"Would  recognize  that  development  contractor  should/could 
provide  some  of  the  elements." 


"I  don't  think  design  interface  is  applicable." 

"Add  Security  Support." 

"For  many  of  the  elements  the  support  required  for  the 
development  is  determined  separately  from  that  for 
maintenance.  These  include:  Manpower,  Training,  Computer 
Resources  and  Facilities.  Maintenance  requirements  in  these 
areas  are  only  part  of  the  total  requirement." 

YOUR  NEW  COMMENTS: 


**22.  Are  there  any  elements  in  the  logistics  support 
planning  of  automated  cartographic  or  photogrammetric 
systems  not  addressed  by  the  LSMM  model?  Please  specify  and 
describe  what  these  elements  are. 

ROUND  1  COMMENTS: 

"Contractor  interface  important  where  equipments  are  very 
unique." 

"Highly  dependent  on  communication  -  narrow  band  and  wide 
band,  fiber  optics,  ethernet,  and  switching  and  transmission 
gear;  may  need  a  separate  communication  category." 

"ILS  documentation  usually  contains  a  safety  chapter  or 
annex.  Safety  needs  to  be  addressed  in  your  model,  but  I  am 
not  sure  how  because  it  cuts  across  most  of  your 
categories . " 

"This  model  fails  to  recognize  the  fundamental  nature  of 
these  systems,  which  is  that  they  are  a  part  of  a  production 
process .  These  processes  and  the  systems  necessarily  change 
and  evolve  as  improvements  are  made  to  increase  productivity 
and  as  new  or  changed  products  are  introduced  into  the 
production  line.  Automated  cartographic  or  photogrammetric 
systems  evolve  during  their  life  cycle  so  that  they  are 
often  very  different  from  their  initial  configuration  and 
use  in  operation.  Changes  are  made  in  both  system  hardware 
and  software  to  effect  this  evolution.  The  ability  to 
constantly  modify  and  improve  applications  software  is 
particularly  crucial  to  the  production  use  of  these  systems. 
Applications  software  is  very  much  an  extension  of  the 
cartographer  or  photogrammetrist .  It  is  probably  not  useful 
or  meaningful  to  try  to  separate  conceptually  software 
maintenance  from  software  development  or  enhancement.  The 


basic  problem  with  the  model  presented  in  that  it  is  linear, 
when  the  actual  life  cycle  for  these  systems  involves  an 
iterative  development  and  operational  cycle." 

"Should  security  be  added  as  an  additional  logistics 
element?" 

"If  you  include  DMA's  traditional  "transition"  events  then 
you  are  missing  Demonstrations  (to  prove  system  meets 
requirements)  and  Exercise  and  Rehearsals  (to  familiarise 
workers  with  the  system  prior  to  production." 

"Security  is  a  major  issue  and  requires  special  attention. 
Calibration  is  a  significant  element  on  some  systems." 

YOUR  NEW  COMMENTS: 


**23.  To  what  extent  does  DMA  employ  ILS  in  logistics 
support  requirements  planning  of  automated  cartographic  or 
photogrammetric  systems? 

ROUND  1  MEAN:  2.759 

YOUR  NEW  RESPONSE: 

1  2 
Not  At 
All 


3 

To  A 

Moderate 

Extent 


5 

To  A 

Very  Great 
Extent 
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TOPIC  3:  ILS  ELEMENTS  AND  LIFE  CYCLE  DETERMINATION  &  DESIGN 
REQUIREMENTS 

For  questions  numbered  24  through  43  please  reference 
Attachment  #2  and  #3.  Life  cycle  phases  are  defined  in 
Attachment  #2  and  ILS  elements  are  defined  in  Attachment  |3 
to  provide  assistance  in  answering  questions  24  through  43. 

For  each  ILS  element  a  list  of  factors  unique  to 
automated  cartographic  or  photogrammetric  systems  was 
collated  from  Round  1  responses.  Review  each  list  and 
provide  the  three  most  important  factors,  for  each  ILS 
element,  that  are  unique  to  automated  cartographic  or 
photogrammetric  systems.  If  required,  please  complete  the 
questions  concerning  when  in  the  life  cycle  of  automated 
cartographic  or  photogrammetric  systems  should  determination 
&  design  requirements  for  each  ILS  element  take  place. 


**24.  Maintenance  Planning:  The  following  list  of 
maintenance  planning  factors  unique  to  automated 
cartographic  or  photogrammetric  systems  was  collated  from 
Round  1  responses.  Please  review  the  list  and  provide  the 
three  most  important  maintenance  planning  factors  unique  to 
automated  cartographic  or  photogrammetric  systems. 

ROUND  1  RESPONSES: 

1.  Adaptive  hardware/software  maintenance 

2.  Corrective  hardware/software  maintenance 

3.  Critical/preliminary  design  review  of  maintenance 
documents 

4.  Determ  ae  whether  maintenance  support  will  be  in-house 
or  contract 

5.  Failure  rates  hard  to  predict  due  to  changing  production 
priorities  for  systems 

6.  Lack  of  application  of  standard  techniques  or 
terminology 

7.  Logisticians  must  be  involved  in  system  development  to 
adequately  plan  for  maintenance  support 

8.  Maintenance  of  electrical  boards 

9.  Maintenance  planning  should  be  a  continuous  process 
throughout  the  life  cycle  of  these  systems 

10.  Maintenance  security  issues  associated  with 
volatile/non-volatile  memory 

11.  Must  conform  to  existing  logistics  maintenance  planning 
guidelines 

12.  No  unique  factors-same  as  any  workstation  automated 
system 

13.  On  call  maintenance  procedures  should  be  established 

14.  Plan  for  possible  default  by  contractors  after  turnover 

15.  Preventive  hardware/software  maintenance 

16.  Preliminary  design  review  of  maintenance  documents 

17.  Rapid  obsolescence 

18.  Sensitive  security  requirements 

19.  Special  skills  to  maintain  unique  equipment  (Automatic 
correlation  equipment,  optics) 


**Please  provide  the  numbers  of  the  three  most  important 
maintenance  planning  factors. 


* 


3 


**25.  At  what  stage  of  the  automated  systems'  life  cycle 
should  maintenance  planning  requirements  determination  and 
design  take  place? 

ROUND  1  MEAN:  2.214 

YOUR  NEW  RESPONSE: 


1 

Conceptual 

Phase 


2 

Advanced 

Development 

Phase 


3 

Detailed 

Design 

Phase 


4 

Production 

Phase 


5 

Operational 

Use 


**26.  Manpower  and  Personnel :  The  following  list  of 
manpower  and  personnel  factors  unique  to  automated 
cartographic  or  photogrammetric  systems  was  collated  from 
Round  1  responses.  Please  review  the  list  and  provide  the 
three  most  important  manpower  and  personnel  factors  unique 
to  automated  cartographic  or  photogrammetric  systems. 

ROUND  1  RESPONSES: 

1.  Ability  to  retain  competent  personnel 

2.  Additional  personnel  in  communication,  optics,  and 
unique  electronics 

3.  Cadre  of  personnel  should  remain  with  the  system  from 
development  through  operational  use 

4.  Education  and  technical  background  of  users 

5.  Fatigue  and  boredom 

6.  Lead  time  for  security  clearances 

7.  Limited  skilled  personnel  to  maintain  hardware  and 
software 

8.  Operational  planning  the  key  driver  of  manpower 
requirements 

9.  Physical/technical  skills  required  for  photogrammetric 
work 

10.  Professional  cartographic  work  force 

11.  Proper  planning  for  staffing  the  automated  systems 

12.  Require  that  more  than  two  hardware  maintenance 
personnel  have  knowledge  of  any  one  system 

13.  Stereo  acuity 

14.  Systems  need  to  be  designed  and  maintained  to  meet 
requirements  of  production  manpower  and  personnel 


**Please  provide  the  numbers  of  the  three  most  important 
manpower  and  personnel  factors. 
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**27.  At  what  stage  of  the  automated  systems'  life  cycle 
should  manpower  and  personnel  requirements  determination  and 
design  take  place? 

ROUND  1  MEAN:  2.552 


YOUR  NEW  RESPONSE: 


1 

Conceptual 

Phase 


2 

Advanced 

Development 

Phase 


3 

Detailed 

Design 

Phase 


4 

Production 

Phase 


5 

Operational 

Use 


ft 


**28.  Supply  Support:  The  following  list  of  supply  support 
factors  unique  to  automated  cartographic  or  photogrammetric 
systems  was  collated  from  Round  1  responses.  Please  review 
the  list  and  provide  the  three  most  important  supply  support 
factors  unique  to  automated  cartographic  or  photogrammetric 
systems . 

ROUND  1  RESPONSES: 

1.  Adequate  shelf  stock  of  required  parts 

2.  COTS  equipment  has  impact  on  the  provisioning  process, 
proprietary  data  rights,  and  basic  supply  support 
concept 

3.  Dependent  on  local  purchase  due  to  unique  spares  not 
being  stocked  in  DOD  supply  system 

4.  Hardware  life  cycle  is  over  before  systems  are  in  full 
production 

5.  Initial  spare  start-up  quantities 

6.  Large  computer  maintenance  contracts 

7.  Optical  components 

8.  Staging  of  spares 

9.  Steady  state  life  cycle  spare  requirements 

10.  Tapes,  removable  drives,  plotter  pens,  plotter  paper 

11.  Unique  systems  need  unique  spares,  requiring  long  lead 
time 


**Please  provide  the  numbers  of  the  three  most  important 
supply  support  factors. 


4 
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29.  At  what  stage  of  the  automated  systems'  life  cycle 
should  supply  support  requirements  determination  and  design 
take  place? 

ROUND  1  CONSENSUS:  68%  AGREE  ON  DETAILED  DESIGN  PHASE. 


**30.  Support  Equipment:  The  following  list  of  support 
equipment  factors  unique  to  automated  cartographic  or 
photogrammetric  systems  was  collated  from  Round  1  responses. 
Please  review  the  list  and  provide  the  three  most  important 
support  equipment  factors  unique  to  automated  cartographic 
or  photogrammetric  systems. 

ROUND  1  RESPONSES: 

1.  Calibration  of  support  and  test  equipment 

2.  Calibration  of  photogrammetric  equipment  is  critical 

3.  Design  for  easily  replaced  logical  units,  which  are 
easily  repaired 

4.  Electronic  circuit  board  testing  and  analysis  equipment 

5.  Identification  of  support  and  test  equipment 

6.  Infreometers 

7.  Optical  alignment  equipment 

8.  Resolution  targets 

9.  Screen  luminance  measuring  equipment 

10.  Signal  generators 

11.  Unique  IBM  gear 


**Please  provide  the  numbers  of  the  three  most  important 
support  equipment  factors. 


31.  At  what  stage  of  the  automated  systems'  life  cycle 
should  support  equipment  requirements  determination  and 
design  take  place? 

ROUND  1  CONSENSUS:  68%  AGREE  ON  DETAILED  DESIGN  PHASE. 


**32.  Technical  Data:  The  following  list  of  technical  data 
factors  unique  to  automated  cartographic  or  photogrammetric 
systems  was  collated  from  Round  1  responses.  Please  review 
the  list  and  provide  the  three  most  important  technical  date 
factors  unique  to  automated  cartographic  or  photogrammetric 
systems . 

ROUND  1  RESPONSES: 

1.  Application  software  should  be  included  in  Technical 
data 

2.  Communicate  with  contractor  when  developing  manuals  to 
be  used  for  production  and  maintenance 

3.  Configuration  items  index 

4.  COTS  equipment  has  impact  on  ability  to  procure  adequate 
technical  data  due  to  proprietary  claims 

5.  Critical /Prel iminary  design  review  of  OSM  and  Logistics 
Plan  Equipment  operations  manuals 

6.  Hardware  maintenance  manuals 

7.  Incorporate  operations  manuals  into  technical  data 
requirements 

8.  Lists  of  delivered  hardware  and  software  items 

9.  Maintenance  and  diagnostics  documentation 

10.  Mathematical  models 

11.  Monitor  -luminance  and  resolution 

12.  Quality  assurance 

13.  Schematics 

14.  Software  documentation  and  interface  manuals 

15.  Stage  calibration  software 

16.  Technical  data  should  be  available  on  microfiche 

17 .  Users  Guides 


**Please  provide  the  numbers  of  the  three  most  important 
technical  data  factors. 


33.  At  what  stage  of  the  automated  systems'  life  cycle 
should  technical  data  requirements  determination  and  design 
take  place? 

ROUND  1  CONSENSUS:  62%  AGREE  ON  DETAILED  DESIGN  PHASE. 
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**34.  Training :  The  following  list  of  training  factors 
unique  to  automated  cartographic  or  photogrammetric  systems 
was  collated  from  Round  1  responses.  Please  review  the  list 
and  provide  the  three  most  important  training  factors  unique 
to  automated  cartographic  or  photogrammetric  systems. 

ROUND  1  RESPONSES: 

1.  Analytical  thinking 

2.  Critical/preliminary  design  review  of  training  plans 

3.  Feature  identification  and  extraction 

4.  Fine  motor  skills  training 

5.  Implement  computer-assisted  training 

6.  In  house  hardware/software  training  must  be  in  depth 

7.  Need  prerequisite  training  to  determine  level  of 
readiness 

8.  Need  preparatory  training 

9.  Need  segment  specific  training 

10.  Need  full-up  production  training 

11.  Need  follow-on  training 

12.  Require  more  than  two  maintenance  personnel  be  trained 
for  any  given  system 

13.  Require  contractor  to  train  more  DMA  personnel  initially 
for  support  activities 

14.  Requirement  for  computer  skills  in  addition  to 
cartographic  skills 

15.  Self  paced  training  courses  (Auto  tutorials) 

16.  Software  maintenance  requires  training  in 
photogrammetric  skills 

17.  Training  should  be  developed  and  executed  by  experienced 
cartographic  and  photogrammetric  personnel  because 
training  often  is  developed  from  scratch 


**Please  provide  the  numbers  of  the  three  most  important 
trailing  factors. 


35.  At  what  stage  of  the  automated  systems'  life  cycle 
should  training  requirements  determination  and  design  take 
place? 


ROUND  1  CONSENSUS:  74%  AGREE  ON  DETAILED  DESIGN  PHASE. 


**36.  Computer  Resources  Support:  The  following  list  of 
computer  resources  support  factors  unique  to  automated 
cartographic  or  photogrammetric  systems  was  collated  from 
Round  1  responses.  Please  review  the  list  and  provide  the 
three  most  important  computer  resources  support  factors 
unique  to  automated  cartographic  or  photogrammetric  systems. 

ROUND  1  RESPONSES: 

1.  Automated  support  and  test  equipment 

2.  COTS  packages  should  be  used  in  lieu  of  special  purpose 
applications  software 

3.  Diagnostic  software 

4.  Flexibility,  expandability,  peak  capacity,  and  fault 
isolation  to  LRU 

5.  Hardware  and  software  should  be  considered  as  separate 
issues 

6.  Image  processing 

7.  Need  a  development  test  facility  to  test  software  code 
fixes  and  upgrades  instead  of  tying  up  production 
equipment 

8.  Operating  system  upgrades  should  be  possible 

9.  Support  for  multiple  brands  of  computers 


**Please  provide  the  numbers  of  the  three  most  important 
computer  resources  support  factors. 


**37.  At  what  stage  of  the  automated  systems'  life  cycle 
should  computer  resources  support  requirements  determination 
and  design  take  place? 

ROUND  1  MEAN:  2.400 


YOUR  NEW  RESPONSE: 


1 

Conceptual 

Phase 


2 

Advanced 
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Phase 


3 
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5 
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**38.  Faci 1 ities :  The  following  list  of  facilities  factors 
unique  to  automated  cartographic  or  photogrammetric  systems 
was  collated  from  Round  1  responses.  Please  review  the  list 
and  provide  the  three  most  important  facilities  factors 
unique  to  automated  cartographic  or  photogrammetric  systems. 

ROUND  1  RESPONSES: 

1.  Access  ramps 

2.  Clean  environment 

3.  Designed  for  human  occupation  not  just  machines 

4.  Electrical  requirements 

5.  Facility  preparation  takes  too  long  creating  disconnect 
between  development  and  facility  requirements 

6.  Fire  protection  requirements 

7.  Humidity  control 

8.  Lighting  requirements 

9.  Noise  reduction  methods 

10.  Personal  storage  areas 

11.  Raised  floor  requirements 

12.  Security  emanations 

13.  Space  requirements 

14.  Special  handling  requirements 

15.  Static  electricity  reduction  techniques 

16.  Temperature  control 

17.  Vibration  control 

18.  Weight  requirements 


**Please  provide  the  numbers  of  the  three  most  important 
facilities  factors. 


**39.  At  what  stage  of  the  automated  systems'  life  cycle 
should  facilities  requirements  determination  and  design  take 
place? 

ROUND  1  MEAN:  2.400 


YOUR  NEW  RESPONSE: 
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**40.  Packaging.  Handling  &  Transportation:  The  following 
list  of  packaging,  handling,  &  transportation  factors  unique 
to  automated  cartographic  or  photogrammetric  systems  was 
collated  from  Round  1  responses.  Please  review  the  list  and 
provide  the  three  most  important  packaging,  handling,  & 
transportation  factors  unique  to  automated  cartographic  or 
photogrammetric  systems. 

ROUND  1  RESPONSES: 

1.  Air  ride  trailers 

2.  Equipment  size  and  weight 

3.  Mode  of  transportation 

4.  Need  a  special  re-usable  container  for  boards,  power 
supplies,  etc 

5.  Packaging  materials  for  optical  equipment 

6.  Personnel  resources 

7.  Security  of  classified  items 

8.  Shipment  of  initial  equipment  from  contractor  to 
customer 

9.  Shipment  of  component  items  from  repair  depot  to 
customer 

10.  Storage  facilities 


**Please  provide  the  numbers  of  the  three  most  important 
packaging,  handling,  &  transportation  factors. 


41.  At  what  stage  of  the  automated  systems'  life  cycle 
should  packaging,  handling  &  transportation  requirements 
determination  and  design  take  place? 

ROUND  1  CONSENSUS:  71%  AGREE  ON  DETAILED  DESIGN  PHASE. 
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**42.  Design  Interface:  The  following  list  of  design 
interface  factors  unique  to  automated  cartographic  or 
photogrammetric  systems  was  collated  from  Round  1  responses. 
Please  review  the  list  and  provide  the  three  most  important 
design  interface  factors  unique  to  automated  cartographic  or 
photogrammetric  systems. 

ROUND  1  RESPONSES: 

1.  Function  of  overall  system  design 

2.  Kean  time  between  failure 

3.  Mean  time  to  repair 

4.  Mean  time  to  restore 

5.  Need  to  incorporate  this  element  early  in  the  life  cycle 

6.  Reliability/Maintainability  requirements 

7.  Useful  system  life 


**Please  provide  the  numbers  of  the  three  most  important 
design  interface  factors. 


**43.  At  what  stage  of  the  automated  systems'  life  cycle 
should  design  interface  requirements  determination  and 
design  take  place? 

ROUND  1  MEAN:  2.259 

YOUR  NEW  RESPONSE: 
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**44.  How  well  does  DMA  apply  ILS  or  any  other  logistics 
support  planning  process  to  automated  cartographic  or 
photogrammetric  systems?  Please  describe  any  improvements 
which  need  to  be  made  to  this  process.  How  would  you 
suggest  these  improvements  be  accomplished? 

ROUND  1  COMMENTS: 

"Currently,  the  planning  process  is  not  done  as  soon  as  it 
should  be-  on  small  procurements  it  is  an  afterthought. 
Personnel  need  to  be  trained  to  think  ILS  during 
development . " 
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"Very  poorly.  There  is  no  integrated  approach  to  logistics 
support  planning.  Individual  elements  are  addressed 
separately  and  some  of  those  elements  are  not  even 
considered  part  of  the  ILS  process.  The  only  real  elements 
DMA  addresses  as  part  of  logistics  support  are  maintenance 
supply,  and  support  and  test  equipment.  The  other  elements 
are  not  really  recognized  as  part  of  the  ILS  process.  DMA 
needs  to  place  more  emphasis  on  integrating  all  of  the 
elements  into  a  comprehensive  and  cohesive  ILS  program.  More 
emphasis  needs  to  placed  on  design  interface  in  order  to 
influence  design  for  supportabi 1 ity  in  the  early  phases  of 
system  development.  Additionally,  the  program  managers  must 
be  made  responsible  for  considering  life  cycle  support,  and 
therefore  ILS,  during  system  design  and  manufacture.  The 
PM's  must  recognize  their  responsibility  for  integrating  .US 
as  one  of  the  program  management  elements." 

"DMA  needs  more  involvement  during  the  Advanced  Development 
and  Detailed  Design  Phases  in  order  to  do  a  better  job  of 
supporting  cartographic  and  photogrammetric  systems. 

"From  my  perspective,  DMA  does  not  do  a  good  job  of 
planning. " 

"Poorly-  we  are  at  least  one  year  behind  but  we  have 
contractors  in  place  to  support  until  we  have  our  sea  legs." 

"Generally,  in  the  past  the  logistics  support  for  production 
systems  has  been  adequate  for  production  needs,  which  would 
indicate  some  planning  process.  Production  departments 
generally  supplied  their  own  software  maintenance. 
Reorganizations  of  maintenance  responsibilities  and 
increased  reliance  of  contractor  maintenance  in  recent  years 
has  generally  lead  to  a  decrease  in  the  level  of 
satisfaction  on  the  production  organizations  in  part." 


"I  saw  little  evidence  of  an  ILS  approach  to  our  past 
systems.  I  do  see  an  attempt  to  provide  ILS  to  our  new 
systems.  Unfortunately,  obstacles  are  constantly  being 
placed  in  our  way.  Logistics  is  viewed  as  obstacle  by  many 
of  the  COTR  type  people.  We  need  more  influence  earlier  in 
the  system  design  phases." 

"Haphazardly-  not  enough  firm  financial  commitment  to 
adequately  support  a  system.  There  needs  to  be  a  Logistics 
Directorate  whose  sole  responsibility  is  maintenance  without 
interference  from  the  Centers  or  other  departments  with  the 
exception  of  communication  during  times  of  crisis.  DMA  needs 
to  get  a  vision  of  maintenance  as  supporting  production 
rather  than  "fix  it  if  it  breaks".  I  would  call  a  strong 
logistics  organization  in  DMA  a  safeguard  on  the  vast 
investment  the  government  has  made  in  DMA." 


"DMA  appears  to  do  a  great  deal  of  planning  but  does  not  do 
a  good  job  of  implementing  the  plan.  Maintenance  people  are 
often  not  adequately  trained  to  support  the  system,  spare 
parts  are  often  not  maintained,  equipment  status  reports  are 
not  adequately  reviewed  or  trend  analysis  performed  and 
preventive  maintenance  is  often  not  performed.  I  would 
review  the  completed  DMA  support  function  from  top  to  bottom 
and  execute  a  plan  to  raise  the  level  of  performance. 

"Very  poorly-  Didn't  apply  the  ILS  early  enough.  Minimal 
planning  in  early  stages.  Now  we  are  playing  catch  up 
through  the  RFC  process." 

"Fair-  More  responsiveness/accountable  to  Production  Center 
requirements  with  Cost  Benefit  Analysis  of  what  work/$  = 
most  beneficial  to  DMA,  not  who  has  what  training  and  is 
available  or  what  crisis  happens  to  call  for  $  when  $  are 
available. " 

"Varies-  MARK  85  and  MARK  90  ILS  very  well  done.  Piecemeal 
buys  are  of  great  variety,  some  good  and  some  non-existent." 

"The  one  area  that  DMA  historically  has  ignored  is  the 
application  of  ILS  during  the  conceptual  phase  of 
development  rather  than  at  the,  too  typically, 
unfortunately,  operational  phase.  MARK  85  and  MARK  90 
systems  are  examples  of  how  DMA  is  changing  with  respect  to 
ILS.  DMA  needs  to  follow  the  formal  processes  of  the  life 
cycles  of  systems  in  a  more  rigorous  manner  for  all  systems, 
as  is  being  done  for  MARK  90." 

"I  believe  that  DMA  does  a  reasonably  good  job  of  dealing 
with  systems  development  and  operation.  However,  we  do  not 
approach  it  in  the  same  way  as  espoused  by  the  OSAF.  In 
recent  times  we  have  had  agency  directors  who  have  tried  to 
dictate  organizational  and  process  changes  to  get  more  in 
line  and  I  have  yet  to  see  the  benefits-  perhaps  somewhere 
in  the  future  they  will  pay  off." 

"DMA  hasn't  done  this  very  well  in  the  past.  As  a  result, 

DMA  Director  is  creating  a  Directorate  for  Acquisition, 
Installation,  and  Logistics.  ADD/Logistics  includes  four 
divisions:  Logistics  Plans,  Distribution,  Maintenance,  and 
Supply.  This  will  eliminate  many  of  DMA's  fragmented 
logistics  activities  and  result  in  a  proactive  consolidated 
logistics  organization  that  is  more  capable  of  developing, 
planning,  and  implementing  true  ILS." 

"He  take  too  long  to  start  the  process,  as  a  result  ILS 
doesn't  do  what  it  is  intended  to  do  (decisions  have  already 
been  made)." 
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"We  have  done  reasonably  well  with  the  Digital  Production 
System  and  subsequent  large  procurements." 

"ILS  is  a  major  player.  Systems  could  not  be  delivered  or  v» 

maintained,  facilities  would  not  be  adequate,  and  personnel 
wouldn't  be  trained  without  ILS.  ILS  has  always  been  a  fuzzy 
area  that  I  always  relied  on  a  logistics  person  to  work.  I 
provided  the  logistics  office  the  technical  specs  of  what 
was  needed  by  when,  and  they  always  came  through."  * 

"Not  very  well-  The  MARK  90  development  is  trying  to 
establish  system  level  requirements  documents  for  ILS  which 
will  help.  Problem  is  that  content  requirements  of  the 
documents  are  overlooked.  Biggest  improvement:  Start 
planning  earlier." 

YOUR  NEW  COMMENTS: 


* 
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LOGISTICS  SYSTEMS  MANAGEMENT  MATRIX  (LSMM) 


J  Research  has  suggested  that  a  model  such  as  the 

Logistics  Systems  Management  Matrix  (LSMM)  below  forms  the 
basis  for  proper  implementation  of  any  given  system.  The 
LSMM  is  a  three-dimensional  model  with  principles  of 
*  management  on  the  "X"  axis,  design  or  life  cycle  processes 

on  the  "Y"  axis,  and  the  ILS  elements  on  the  "Z"  axis. 
Please  use  Attachment  #1  as  a  reference  in  answering 
questions  11  through  22. 
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The  Logistics  Systems  Management  Matrix 
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DESIGN  OR  LIFE  CYCLE  PHASES 


I 

1.  Conceptual  Phase:  This  phase  is  where  the  product  or 

service  system  is  defined,  acquisition  requirements  are 
determined,  production  and  operational  support  requirements  ^ 

are  identified  and  logistics  support  planning  is 

established. 

2.  Advanced  Development  Phase:  This  phase  identifies  and 
defines  the  product  configuration.  It  deals  with  product 
objectives,  performance  limits,  areas  of  risk,  alterations 
and  acquisition  methodologies.  During  this  phase,  logistics 
support  analysis  is  conducted  to  verify  design  concepts, 
followed  by  the  initiation  of  the  logistics  support  plan. 

3.  Detailed  Design  Phase:  This  phase  includes  the  design 
and  test  of  the  product.  It  consists  of  design  effects  on 
reliability,  maintainability,  sustainability,  human  factors 
issues  and  logistics  support.  Engineering  modes  and 
prototypes  are  developed.  At  this  time,  the  formal  logistic 
support  plan  is  completed. 

4.  Production  Phase:  This  phase  consists  of  producing  the 
product  and  its  support,  such  as  test  and  support  equipment, 
spare  parts,  training  personnel,  software,  physical 
distribution  and  transportation,  and  warehousing. 

5.  Operational  Use:  This  phase  covers  all  of  the 
activities  associated  with  consumer  use  and  logistics 
support,  to  include  maintenance  and  sustainability. 

6.  Retirement ;  This  phase  consists  of  all  of  the  actions 
necessary  to  phase  out  and  dispose  of  the  system,  to  include 
waste  management,  recycling  or  other  responsive  actions. 

Fast  obsolescence  systems,  such  as  automated  cartographic  or 
photogrammetric  systems,  require  retirement  planning  early 
in  the  life  cycle  to  ensure  proper  phase-out  of  old  systems 
and  implementation  of  new  systems. 
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ILS  ELEMENTS 


The  fol losing  ILS  definitions  come  from  the  Integrated 
Logistics  Support  Guide  published  by  the  Defense  Systems 
Management  College.  Each  definition  is  a  short  synopsis 
for  that  particular  ILS  element.  Please  use  the  ILS 
definitions  provided  in  Attachment  #2  as  a  reference  in 
answering  Questions  23  through  42. 

1.  Maintenance  Planning:  The  process  conducted  by  the 
Government  and  contractor  to  explore  alternatives  and  to 
develop  the  maintenance  concepts  and  maintenance 
requirements  for  the  life  of  the  material  system. 

Maintenance  planning  is  the  lead  analytical  activity  and 
provides  input  to  the  development  of  all  of  the  remaining 
logistics  support  elements. 

2.  Manpower  and  Personnel :  The  element  which  encompasses 
the  identification  and  acquisition  of  military  and  civilian 
personnel  with  the  skills  and  grades  required  to  operate  and 
support  a  material  system  over  its  lifetime  at  peacetime  and 
wartime  rates. 

3.  Supply  Support:  The  element  which  encompasses  all 
actions  required  to  identify  and  obtain  the  spares  and 
repair  parts  needed  to  support  peacetime  and  wartime 
readiness  objectives. 

4.  Support  Eguipment:  The  element  which  encompasses  all 
equipment  required  to  support  operation  and  maintenance  of 
the  material  system.  This  includes  ground  handling 
equipment,  tools,  metrology  and  calibration  equipment,  test 
equipment,  and  logistic  support  for  the  support  equipment. 

5.  Technical  Data:  The  element  which  encompasses  all 
recorded  information  of  a  scientific  or  technical  nature 
related  to  a  program.  Technical  data  are  written 
instructions  such  as  drawings;  operating  and  maintenance 
manuals;  specifications,  inspection,  test  and  calibration 
procedures;  and  computer  programs  which  guide  personnel 
performing  operations  and  support  tasks. 

6.  Training:  The  element  which  encompasses  all  of  the 
processes,  procedures,  techniques,  training  devices  and 
equipment  used  to  train  personnel  to  operate  and  support  a 
material  system.  Examples  include  individual  and  group 
training;  new  equipment  training;  initial,  formal  and  on- 
the-job  training;  and  logistic  support  planning  for  training 
equipment . 
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7.  Computer  Resources  Support:  The  element  is  defined  as 
all  computer  equipment,  software,  associated  documentation, 
contractual  services,  personnel,  and  supplies  needed  to 

operate  and  support  an  embedded  computer  system.  -» 

8.  Faci 1 ities :  The  element  which  encompasses  those  real 

property  assets  required  to  support  the  material  system,  and 
the  studies  which  define  types  of  facilities  or  facility 
improvements,  locations,  space  needs,  etc.  * 

9.  Packaging,  Handling  &  Transportation:  The  element  which 
includes  the  characteristics,  action  and  requirements 
necessary  to  ensure  the  capability  to  transport,  preserve, 
package,  and  handle  all  equipment  and  support  items. 

10.  Design  Interface:  The  element  which  is  an  interactive 
relationship  of  logistics-related  design  parameters,  such  as 
R&M,  to  readiness  and  logistic  support  resource 

,'^qui  r em^nts . 
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Appendix  H :  Round  Two  Delphi  Survey  Comments 


Basic  Characteristics  of  the  LSMM  Model . 

Basic  Structure . 

How  would  you  change  the  LSMM? 

"Disagree  with  adding  resources  =  $$.  Resources  are  a 
limiting  factor  to  the  individual  categories  as  well  as  to 
the  model  as  a  whole.  "Transition  is  not  a  separate 
logistics  step;  one  could  put  a  "transition"  step  between 
every  life  cycle  phase.  Test  equipment  should  not  be  a 
separate  element  because  it  is  a  type  of  "support 
equipment".  DMA  should  use  DOD  terminology  rather  than  its 
own.  The  LSMM  is  a  logistics  systems  model .  Models  can  be 
as  simple  or  as  complex  as  we  want.  Recommend  the  following 
changes:  (1)  under  "controlling"  add  "including  budgeting"; 
(2)  on  the  "Z"  axis  add  a  separate  software  category;  (3) 
realign  "Z"  axis  (bottom  to  top):  maintenance  planning, 
design  interface,  technical  data,  software,  computer 
resources  support,  support  equipment,  supply  support, 
facilities,  packaging,  handling,  storage,  &  transportation, 
manpower  and  personnel,  and  training." 

"A  number  of  comments  were  made  on  the  life  cycle  phases. 
Generally  speaking,  I  think  we  should  be  cons.stent  with  the 
phases  designated  in  DODD  5000.1,  which  is  being  revised. 
Under  the  revised  Directive  DMA  will  no  longer  be  able  to 
make  claims  of  exemption.  The  DPS  program  has  been  under 
scrutiny  as  a  result  of  the  claim  of  exemption,  as  noted  by 
the  DOD  IG.  Although  I'm  not  sure,  there  may  be  provisions 
for  product  improvement  in  the  new  DODD  5000.1." 

"Add  security  considerations  to  the  "Z"  axis." 

"I  would  agree  that  transition  and  $$  resources  should  be 
added  to  the  model.  The  glass  is  half  full  or  half  empty 
with  respect  to  maintenance  planning  -  it  should/could  be 
included  in  overall  planning . " 

"Add  delivery,  installation,  test  and  acceptance,  and 
operational  refinement  to  "Y"  axis." 

"There  is  nothing  wrong  with  the  model.  It  is  only  how  the 
model  is  implemented  that  is  the  problem." 

"If  your  intention  is  to  use*  the  generic  LSMM  for  DMA 
systems,  then  the  majority  of  the  round  one  comments  need  to 
be  accommodated." 


"Add  budgeting  to  the  "X"  axis  and  delete  maintenance 
planning  and  computer  resources  support  and  replace  with 
hardware  maintenance  and  software  support .  The  definition 
of  software  support  should  include  system  and  applications 
software  maintenance,  COTS  upgrades,  and  the  maintenance  of 
documentation.  In  general,  the  model  does  not  reflect  the 
situation  that  we  have  typically  with  DMA  automated 
cartographic  or  photogrammetric  systems  which  is  one  of 
continual  change  and  evolution  over  the  lifetime  of  a 
production  system.  Typically,  the  hardware  changes  slowly, 
but  there  is  usually  a  fairly  steady  and  rapid  change  in 
software,  often  punctuated  by  major  software  changes  in 
capability  and/or  replacement/upgrade  of  computer 
components.  Such  system  components  as  stereoplotters  or 
scanners  may  change  little  over  a  20  year  period." 

"Change  "Advanced  Development  Phase"  to  "Development  Phase". 
Add  software  and  hardware  maintenance  as  separate  items  to 
the  "Z"  axis." 


Design  or  Life  Cycle  Processes . 


How  would  you  change  the  design  or  life  cycle  phases 
proposed  in  the  LSMM? 

"Waiting  until  the  later  phases  before  considering  logistics 
support  planning  is  an  invitation  to  inefficient  and 
ineffectiveness  at  best  and  disaster  at  worst.  Past  records 
within  DOD  clearly  demonstrate  that  life  cycle  logistics 
support  costs  often  exceed  initial  purchase  cost  by  3  to  5 
times.  Accordingly,  ILS  must  be  considered  early  and 
continue  to  be  a  strong  player  throughout  the  process." 

"Production  and  operational  phases  should  be  combined.  Add 
production  improvement  phase.  Add  preliminary  design  phase 
and  transition  to  production  phase." 


'Common  DMA  terms  for  life  cycle  phases  are  as  follows: 


Conceptual 

Advanced  Development 
Detailed  Design 
Production 
(No  Counterpart) 
Operational  Use 


Design  Concept 
Preliminary  Design 
Critical  Design 
Build 

Transition 

Production  Operations 
Excessing 


Retirement  = 

Because  DMA  is  a  production  organization,  the  use  of  the 
word  production  when  associated  with  cartographic  or 
photogrammetric  systems  is  usually  synonymous  with 
"operational  use"  in  the  LSMM  model .  This  causes 
miscommunication  between  ILS  personnel  and  DMA  end  users. 
"Retirement"  -  A  comment-  DMA  has  systems  that  have  been  in 
production  operations  for  over  20  years.  They  cannot  be 
considered  as  fast  obsolescence  systems." 
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"I  would  agree  with  the  incorporation  of  a  transition  phase 
and  the  breakout  of  the  operational  phase." 

Integrated  Logistics  Support  (ILS)  Elements . 

Which  of  the  ILS  elements  are  not  applicable  to 
automated  cartographic  or  photogrammetric  systems?  Why 
are  they  not  applicable? 

"None  -  The  least  important  call  is  tough  even  packaging, 
handling,  storage,  &  transportation  is  important  with  a 
planning  factor  of  1000  failed  LRU's  a  month,  most  of  which 
must  be  shipped  out  for  depot  maintenance." 

"Some  of  the  respondents  don't  seem  to  comprehend  that  ILS 
is  a  process,  not  iust  an  organization.  The  process  and  the 
ILS  manager  implementing  the  process  integrates  the  separate 
logistics  elements  into  a  cohesive  and  comprehensive 
logistics  support  program.  The  ILS  manager  or  the  ILS 
office  does  not  necessarily  have  direct  authority  and 
responsibility  for  each  and  every  one  of  the  separate 
logistics  elements.  It  is  important  for  each  of  those 
elements  to  recognize  they  play  a  part  in  the  ILS  process 
and  the  impact  their  decisions  may  have  on  the  other 
logistics  support  elements." 

"Perhaps  the  LSMM  should  be  tailored  to  DMA  elements.  Also, 
ILS  concepts  are  not  uniformly  understood  throughout  DMA." 

"There  should  be  a  distinction  between  operations  manpower 
and  maintenance  manpower.  The  determination  of  the  numbers 
of  operations  manpower  is  part  of  the  production  programming 
process  and  is  driven  by  the  required  production  rates. 
Operations  manpower  requirements  determination  is  not  part 
of  the  ILS  process." 

"Design  interface  still  seems  the  least  well  defined." 

"All  parts  of  ILS  are  important,  but  unless  we  plan  to  move 
systems,  packaging,  handling,  storage,  &  transportation  is 
probably  the  least  important.  I  think  some  participants 
missed  the  point  concerning  manpower  and  personnel  -  in  ILS 
this  usually  refers  to  the  people  maintaining  the  system, 
not  the  operators.  Remember,  this  is  a  logistics  model. 
There  is  another  whole  world  that  analyzes  what  the  system 
does,  how  does  it  work,  and  who  operates  it." 

How  would  you  change  any  of  the  ILS  elements  proposed 
in  the  LSMM? 

"Assigning  ILS  tasks  to  a  contractor  is  possible,  but  that 
doesn't  eliminate  those  requirements  or  abrogate  our 
responsibility  for  ensuring  ILS  requirements  are  met." 


"ADP  security  needs  to  be  added." 

"Since  some  respondents  didn't  seem  to  understand  design 
interface,  additional  definition  would  probably  be  helpful. 

Computer  Resources  Support  is  also  unclear.  Maybe  it  should 
be  reworded  to  something  more  along  the  line  of  the  "Post 
Deployment  Software  Support"  chapter  in  DMSC's  Mission 
Critical  Computer  Resources  Guide.  Software  support  should  | 

not  be  lumped  under  maintenance  planning." 

"There  should  be  a  distinction  between  the  production  use  of 
a  system  and  system  support.  Training  and  determination  of 
staffing  levels  for  operational  use  of  the  system  is  a 
mission  related  process." 

"If  the  LSMM  is  a  model  to  be  used  throughout  all  of  the 
phases  of  a  system,  then  the  model  should  be  revised.  The 
ILS  elements  are  oriented  toward  the  planning  aspects  of 
logistics  support  rather  than  execution.  Thus,  it  is  hard 
to  identify  in  the  LSMM  an  element  for  directing  hardware 
maintenance  activities,  as  an  example." 

"Security  support  should  be  added." 

Are  there  any  elements  in  the  logistics  support 
planning  of  automated  cartographic  or  photogrammetric 
systems  not  addressed  by  the  LSMM  model?  Please 
specify  and  describe  what  these  elements  are. 

"We  need  to  concern  ourselves  more  with  Total  Quality 
Management  (TQM)  and  Configuration  Management  (CM)." 

"What  is  missing  is  the  fact  that  ILS  puts  a  tremendous 
overhead  on  the  whole  process.  We  must  find  a  way  to  make 
ILS  work  with  minimal  impact." 

"All  of  these  things  can  be  covered  under  the  existing  ILS 
process.  The  process  is  not  equipment  type  specific  and  in 
fact,  command  and  control  systems  frequently  encounter  the 
same  constant  upgrade  of  software  capabilities  There  is 
nothing  in  the  process  that  can't  cover  all  the  comments. 

You  simply  tailor  the  program  accordingly." 

"Add  security  requirements  and  considerations."  + 

"The  model  represents  a  basic  conception  through  death 

linear  process.  Most  of  our  systems  go  through  continual 

cycles  of  growth,  decay,  regeneration,  and  reincarnation 

before  they  finally  die."  * 

"Security  considerations  are  important  and  need  to  be 
included. " 
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Logistics  Support  Planning  Applications  within  DMA . 


How  well  does  DMA  apply  ILS  or  any  other  logistics 
support  planning  process  to  automated  cartographic  or 
photogrammetric  systems?  Please  describe  any 
improvements  which  need  to  be  made  to  this  process. 

How  would  you  suggest  these  improvements  be 
accompl ished? 

"DMA  is  recognizing  the  importance  of  ILS.  The  creation  of 
the  new  Directorate  of  Acquisition,  Installation,  and 
Logistics  should  help  institutionalize  the  process.  The  ILS 
awareness  expressed  by  our  senior  managers  needs  to  be 
nurtured.  Above  all,  ILS  must  be  seen  as  a  process  that 
will  enhance  the  effectiveness  and  efficiency  of  our 
production  systems  through  increased  reliability, 
maintainability,  and  availability." 

"There  is  no  Integrated  Logistics  Support  process  in  place 
for  MARK  85  and  MARK  90!  It  is  piecemeal.  Emphasis  must  be 
placed  on  design  for  supportability ,  so  the  support  process 
is  not  merely  reacting  to  what  has  already  been  locked  in 
during  design.  What  support  we  have  managed  to  provide  has 
been  "in  spite  of"  rather  than  "because  of"  DMA's  degree  of 
attention  toward  logistics  support." 

"DMA  logisticians  must  demonstrate  what  they  can  contribute 
to  the  overall  function  of  DMA." 

"It  is  extremely  difficult  to  rank  most  or  least  important 
on  categories  where  all  the  constituents  are  essential  to 
the  success  of  production  systems  development,  delivery, 
test,  and  production  use." 

"In  general,  life  cycle  costs  are  not  addressed  well  in  DMA. 
The  objective  of  system  developers  is  often  to  minimize 
development  cost  at  the  expense  of  total  system  life  cycle 
costs.  Because  of  the  nature  of  our  mission  and  the 
evolution  of  our  systems  over  what  is  typically  a  very  long 
life  cycle,  flexibility,  adaptability,  and  long  term 
maintainability  should  be  given  more  consideration  in  the 
design  process  than  have  frequently  been  the  case." 

"I  need  to  reiterate  and  augment  a  portion  of  a  previous 
comment:  Program  Managers  must  be  made  responsible  for  the 

life  cycle  support,  and  therefore  ILS,  during  the  system 
design  and  development.  I  would  even  go  further  and  state 
that  Program  Managers  should  be  responsible  for  the  total 
support  during  the  life  cycle  of  a  system." 

"Interesting  perspectives...  Many  think  "earlier"  when  in 
reality  all  you  get  is  less  firmness.  Best  time  to  finalize 
an  ILS  plan  is  when  you  are  building  something." 


"Logistics  is  not  being  accomplished.  Some  maintenance  is 
done,  some  supply  is  done,  a  portion  of  all  logistics 
elements  is  being  done.  No  one  is  looking  at  the  logistics 
umbrella  as  a  comprehensive  integrated  system.  The  failure  1 

to  do  this  will  add  a  minimum  of  10%  to  support  costs." 

"We  simply  haven't  done  a  good  job  here  -  we  constantly  play 

catch  up."  | 

"ILS  has  not  been  formalized  within  DMA.  As  DOD  becomes 
more  demanding  of  ILS,  the  Defense  Agencies  will  and  ^re 
starting  to  give  ILS  more  serious  considerations." 
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Appendix  I :  Delphi  Survey  Respondents 


Defense  Mapping  Agency  Headguarters  ( DMAHO ) 

Mr.  Penman  R.  Gilliam 
Mr.  Mordecai  Z.  Labovitz 
Mr.  George  A.  Pelletiere 
Mr.  James  Sippel 
Ms.  Janice  Wollaber 

Defense  Mapping  Agency  Systems  Center  ( DMASC) 

Mr.  Merle  J.  Biggin 
Mr.  John  W.  Bukoski 
Mr.  Frederick  J.  Doyle,  Jr. 

Mr.  Gerald  M.  Elphingstone 
Mr.  Clyde  F.  Housel 
Lt  Col  Jay  L.  Larson,  USAF 
Mr.  Robert  F.  Seebald 
Mr.  Charles  K.  Shand,  Jr. 

Mr.  Herbert  F.  Smith 
Mr.  Morris  S.  Solomon 
Ms.  Linda  J.  Sullivan 
Mr.  Michael  B.  Wagner 
Mr.  John  Webster 

Defense  Mapping  Agency  Telecommunications  Services  Center 
( DMATSC ) 


Mr.  Bill  Robinson 

Defense  Mapping  Agency  Reston  Center  ( DMARC ) 

Mr.  Darryl  Crumpton 
Mr.  Russell  Gustin 
Col  Peter  G.  O'Neill,  USA 
Mr.  Paul  L.  Peeler,  Jr. 

Defense  Mapping  Agency  Hydrographic /Topographic  Center 
( DMAHTC ) 

Capt  John  E.  Chubb,  USN 
Mr.  Thomas  K.  Coghlan 
Ms.  Maureen  P.  Currie 
Mr.  David  S.  Scopp 
Mr.  Lon  M.  Smith 
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Mr.  Leslie  R.  Kemp 
Mr.  James  R.  Skidmore 
Ms.  Kathleen  M.  Smith 


Appendix  J:  Unique  Factors  of  ILS  Elements 

Maintenance  Planning .  Table  39  lists  the  maintenance 
planning  factors  unique  to  automated  cartographic  or 
photogrammetric  systems  provided  by  the  DMA  experts.  The 
list  was  collated  from  round  one  responses. 

Table  39 

Maintenance  Planning  Factors  Unique 

to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  One  Delphi  Survey 


1.  Adaptive  hardware/software  maintenance 

2.  Corrective  hardware/software  maintenance 

3.  Critical/preliminary  design  review  of  maintenance 
documents 

4.  Determine  whether  maintenance  support  will  be  in-house 
or  contract 

5.  Failure  rates  hard  to  predict  due  to  changing  production 
priorities  for  systems 

6.  Lack  of  application  or  standard  techniques 

7.  Logisticians  must  be  involved  in  system  development  to 
adequately  plan  for  maintenance  support 

8.  Maintenance  of  electrical  boards 

9.  Maintenance  planning  should  be  a  continuous  process 

throughout  the  life  cycle  of  these  systems 

10.  Maintenance  security  issues  associated  with 
volatile/non-volatile  memory 

11.  Must  conform  to  existing  logistics  maintenance  planning 
guidelines 

12.  No  unique  factors-same  as  any  workstation  automated 
system 

13.  On  call  maintenance  procedures  should  be  established 

14.  Plan  for  possible  default  by  contractors  after  turnover 

15.  Preventive  hardware/software  maintenance 

16.  Preliminary  design  review  of  maintenance  documents 

17.  Rapid  obsolescence 

18.  Sensitive  security  requirements 

19.  Special  skills  to  maintain  unique  equipment  (Automatic 
correlation  equipment,  optics) 


Manpower  and  Personnel .  Table  40  lists  the 


manpower/personnel  factors  unique  to  automated  cartographic 
or  photogrammetric  systems  provided  by  the  DMA  experts.  The 
list  was  collated  from  round  one  responses. 


Table  40 

Manpower/Personnel  Factors  Unique 

to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  One  Delphi  Survey 


1.  Ability  to  retain  competent  personnel 

2.  Additional  personnel  in  communication,  optics,  and 
unique  electronics 

3.  Cadre  of  personnel  should  remain  with  the  system  from 
development  through  operational  use 

4.  Education  and  technical  background  of  users 

5.  Fatigue  and  boredom 

6.  Lead  time  for  security  clearances 

7.  Limited  skilled  personnel  to  maintain  hardware  and 
software 

8.  Operational  planning  the  key  driver  of  manpower 
requirements 

9.  Physical/technical  skills  required  for  photogrammetric 
work 

10.  Professional  cartographic  work  force 

11.  Proper  planning  for  staffing  the  automated  systems 

12.  Require  that  more  than  two  hardware  maintenance 
personnel  have  knowledge  of  any  one  system 

13.  Stereo  acuity 

14.  Systems  need  to  be  designed  and  maintained  to  meet 
requirements  of  production  manpower  and  personnel 


Supply  Support .  Table  41  lists  the  supply  support 
factors  unique  to  automated  cartographic  or  photogrammetric 
systems  provided  by  the  DMA  experts.  The  list  was  collated 
from  round  one  responses. 


Table  41 

Supply  Support  Factors  Unique 
to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  One  Delphi  Survey 


1.  Adequate  shelf  stock  of  required  parts 

2.  Commercial  Of f-the-Shel f  (COTS)  equipment  has  impact  on 
the  provisioning  process,  proprietary  data  rights,  and 
basic  supply  support  concept 

3.  Dependent  on  local  purchase  due  to  unique  spares  not 
being  stocked  in  DOD  supply  system 

4.  Hardware  life  cycle  is  over  before  systems  are  in  full 
production 

5.  Initial  spare  start-up  quantities 

6.  Large  computer  maintenance  contracts 

7.  Optical  components 

8.  Staging  of  spares 

9.  Steady  state  life  cycle  spare  requirements 

10.  Tapes,  removable  drives,  plotter  pens,  plotter  paper 

11.  Unique  systems  need  unique  spares,  requiring  long  lead 
time 
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Support  Equipment .  Table  42  lists  the  support 


equipment  factors  unique  to  automated  cartographic  or 
photogrammetric  systems  provided  by  the  DMA  experts.  The 
list  was  collated  from  round  one  responses. 


\ 


Table  42 

Support  Equipment  Factors  Unique 

to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  One  Delphi  Survey 


% 


1.  Calibration  of  support  and  test  equipment 

2.  Calibration  of  photogrammetric  equipment  is  critical 

3.  Design  for  easily  replaced  logical  units,  which  are 
easily  repaired 

4.  Electronic  circuit  board  testing  and  analysis  equipment 

5.  Identification  of  support  and  test  equipment 

6.  Infreometers 

7.  Optical  alignment  equipment 

8.  Resolution  targets 

9.  Screen  luminance  measuring  equipment 

10.  Signal  generators 

11.  Unique  IBM  gear 


> 
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Technical  Data .  Table  43  lists  the  technical  data 


factors  unique  to  automated  cartographic  or  photogrammetric 
systems  provided  by  the  DMA  experts.  The  list  was  collated 
from  round  one  responses. 


Table  43 

Technical  Data  Factors  Unique 
to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  One  Delphi  Survey 


1.  Application  software  should  be  included  in  Technical 
data 

2.  Communicate  with  contractor  when  developing  manuals  to 
be  used  for  production  and  maintenance 

3.  Configuration  items  index 

4.  COTS  equipment  has  impact  on  ability  to  procure  adequate 
technical  data  due  to  proprietary  claims 

5.  Critical/Preliminary  design  review  of  O&M  and  Logistics 
Plan  Equipment  operations  manuals 

6.  Hardware  maintenance  manuals 

7.  Incorporate  operations  manuals  into  technical  data 
requirements 

8.  Lists  of  delivered  hardware  and  software  items 

9.  Maintenance  and  diagnostics  documentation 

10.  Mathematical  models 

11.  Monitor  luminance  and  resolution 

12.  Quality  assurance 

13.  Schematics 

14.  Software  documentation  and  interface  manuals 

15.  Stage  calibration  software 

16.  Technical  data  should  be  available  on  microfiche 

17 .  Users  Guides 


* 
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Training .  Table  44  lists  the  training  factors  unique 


to  automated  cartographic  or  photogrammetric  systems 
provided  by  the  DMA  experts.  The  list  was  collated  from 
round  one  responses. 


Table  44 

Training  Factors  Unique 
to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  One  Delphi  Survey 


1.  Analytical  thinking 

2.  Critical/preliminary  design  review  of  training  plans 

3.  Feature  identification  and  extraction 

4.  Fine  motor  skills  training 

5.  Implement  computer-assisted  training 

6.  In  house  hardware/software  training  must  be  in  depth 

7.  Need  prerequisite  training  to  determine  level  of 
readiness 

8.  Need  preparatory  training 

9.  Need  segment  specific  training 

10.  Need  full -up  production  training 

11.  Need  follow-on  training 

12.  Require  more  than  two  maintenance  personnel  be  trained 
for  any  given  system 

13.  Require  contractor  to  train  more  DMA  personnel  initially 
for  support  activities 

14.  Requirement  for  computer  skills  in  addition  to 
cartographic  skills 

15.  Self  paced  training  courses  (Auto  tutorials) 

16.  Software  maintenance  requires  training  in 
photogrammetric  skills 

17.  Training  should  be  developed  and  executed  by  experienced 
cartographic  and  photogrammetric  personnel  because 
training  often  is  developed  from  scratch 


Computer  Resources  Support .  Table  45  lists  the 


computer  resources  support  factors  unique  to  automated 
cartographic  or  photogrammetric  systems  provided  by  the  DMA 
experts.  The  list  was  collated  from  round  one  responses. 


Table  45 

Computer  Resources  Support  Factors  Unique 

to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  One  Delphi  Survey 


1.  Automated  support  and  test  equipment 

2.  COTS  packages  should  be  used  in  lieu  of  special  purpose 
applications  software 

3.  Diagnostic  software 

4.  Flexibility,  expandability,  peak  capacity,  and  fault 
isolation  to  LRU 

5.  Hardware  and  software  should  be  considered  as  separate 
issues 

6.  Image  processing 

7.  Need  a  development  test  facility  to  test  software  code 
fixes  and  upgrades  instead  of  tying  up  production 
equipment 

8.  Operating  system  upgrades  should  be  possible 

9.  Support  for  multiple  brands  of  computers 


Facilities.  Table  46  lists  the  facilities  factors 


unique  to  automated  cartographic  or  photogrammetric  systems 
provided  by  the  DMA  experts.  The  list  was  collated  from 
round  one  responses. 


Table  46 

Facilities  Factors  Unique 
to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  One  Delphi  Survey 


1.  Access  ramps 

2.  Clean  environment 

3.  Designed  for  human  occupation  not  just  machines 

4.  Electrical  requirements 

5.  Facility  preparation  takes  too  long  creating  disconnect 
between  development  and  facility  requirements 

6.  Fire  protection  requirements 

7.  Humidity  control 

8.  Lighting  requirements 

9.  Noise  reduction  methods 

10.  Personal  storage  areas 

11.  Raised  floor  requirements 

12.  Security  emanations 

13.  Space  requirements 

14.  Special  handling  requirements 

15.  Static  electricity  reduction  techniques 

16.  Temperature  control 

17.  Vibration  control 

18.  Weight  requirements 


Packaging .  Handling.  Storage ,  &_  Transportation.  Table 


47  lists  the  packaging,  handling,  storage,  &  transportation 
factors  unique  to  automated  cartographic  or  photogrammetric 
systems  provided  by  the  DMA  experts.  The  list  was  collated 
from  round  one  responses. 


Table  47 

Packaging,  Handling,  Storage,  &  Transportation  Factors 

Unique  to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  One  Delphi  Survey 


1.  Air  ride  trailers 

2.  Equipment  size  and  weight 

3.  Mode  of  transportation 

4.  Need  a  special  re-usable  container  for  boards,  power 
supplies,  etc 

5.  Packaging  materials  for  optical  equipment 

6.  Personnel  resources 

7.  Security  of  classified  items 

8.  Shipment  of  initial  equipment  from  contractor  to 
customer 

9.  Shipment  of  component  items  from  reoair  derm*  t-n 
customer 

10.  Storage  facilities 


Design  Interface .  Table  48  lists  the  design  interface 
factors  unique  to  automated  cartographic  or  photogrammetric 
systems  provided  by  the  DMA  experts.  The  list  was  collated 
from  round  one  responses. 


1 


f 


Table  48 

Design  Interface  Factors  Unique 
to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  One  Delphi  Survey 


1.  Function  of  overall  system  design 

2.  Mean  time  between  failure 

3.  Mean  time  to  repair 

4.  Mean  time  to  restore 

5.  Need  to  incorporate  this  element  early  in  the  life  cycle 

6.  Reliability/Maintainability  requirements 

7.  Useful  system  life 


i 
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Appendix  K:  Ranked  Unique  Factors  of  ILS  Elements 

Maintenance  Planning .  Table  49  lists  the  ranking  and 
frequency  count  of  maintenance  planning  factors  unique  to 
automated  cartographic  or  photogrammetric  systems  as 
provided  by  the  DMA  experts  in  round  two. 


Table  49 

Maintenance  Planning  Factors  Unique  to 
Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  Two  Delphi  Survey 


Frequency 


1.  Determine  whether  maintenance  support  will  be  14 

in-house  or  contract 

2.  Corrective  hardware/software  maintenance  11 

3.  Maintenance  planning  should  be  a  continuous  10 

process  throughout  the  life  cycle  systems 

4.  Logisticians  involved  in  system  development  8 

to  plan  for  maintenance  support 

5.  Adaptive  hardware/software  maintenance  7 

6.  Critical/preliminary  design  review  of  6 

maintenance  documents 

7.  Special  skills  to  maintain  unique  equipment  4 

(Automatic  correlation  equipment,  optics) 

8.  Preventive  hardware/software  maintenance  4 

9.  Lack  of  application  or  standard  techniques  3 

10.  Maintenance  security  issues  associated  with  2 

volatile/non-volatile  memory 

11.  Failure  rates  hard  to  predict  due  to  changing  1 

production  priorities  for  systems 

12.  Must  conform  to  existing  logistics  1 

maintenance  planning  guidelines 

13.  No  unique  factors-same  as  any  workstation  1 

automated  system 

14.  On  call  maintenance  procedures  established  1 

15.  Plan  for  possible  default  by  contractors  1 

16.  Sensitive  security  requirements  1 

17.  Maintenance  of  electrical  boards  0 

18.  Preliminary  design  review  of  maintenance  0 

documents 

19.  Rapid  obsolescence  0 
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Manpower  and  Personnel .  Table  50  lists  the  ranking  and 
frequency  count  of  manpower/personnel  factors  unique  to 
automated  cartographic  or  photogrammetric  systems  as 
provided  by  the  DMA  experts  in  round  two. 

Table  50 

Manpower/Personnel  Factors  Unique 

to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  Two  Delphi  Survey 


) 

f 


Frequency 


1.  Ability  to  retain  competent  personnel  20 

2.  Limited  skilled  personnel  to  maintain  hardware  10 

and  software 

3.  Systems  need  to  be  designed  and  maintained  to  9 

meet  requirements  of  production  manpower 

and  personnel 

4.  Proper  planning  for  staffing  the  automated  6 

systems 

5.  Education  and  technical  background  of  users  5 

6.  Operational  planning  the  key  driver  of  5 

manpower  requirements 

7.  Additional  personnel  in  communication,  optics,  4 

and  unique  electronics 

8.  Cadre  of  personnel  to  remain  with  the  system  4 

from  development  through  operational  use 

9.  Lead  time  for  security  clearances  3 

10.  Professional  cartographic  work  force  3 

11.  Fatigue  and  boredom  2 

12.  Physical/technical  skills  required  for  2 

photogrammetric  work 

13.  Require  that  more  than  two  hardware  1 

maintenance  personnel  have  knowledge  of  any 

one  system 

14.  Stereo  acuity  1 


) 
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Supply  Support .  Table  51  lists  the  ranking  and 
frequency  count  of  supply  support  factors  unique  to 
i  automated  cartographic  or  photogrammetric  systems  as 

provided  by  the  DMA  experts  in  round  two. 


Table  51 

Supply  Support  Factors  Unique 
to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  Two  Delphi  Survey 


Frequency 


1.  Adequate  shelf  stock  of  required  parts  18 

2.  Unique  systems  need  unique  spares,  requiring  13 

long  lead  time 

3.  Commercial  Of f -the-Shel f  (COTS)  equipment  has  10 

impact  on  the  provisioning  process,  proprietary 

data  rights,  and  basic  supply  support  concept 

4.  Initial  spare  start-up  quantities  8 

5.  Hardware  life  cycle  is  over  before  systems  are  7 

in  full  production 

6.  Dependent  on  local  purchase  due  to  unique  5 

spares  not  being  stocked  in  DOD  supply  system 

7.  Staging  of  spares  5 

8.  Large  computer  maintenance  contracts  4 

9.  Steady  state  life  cycle  spare  requirements  3 

10.  Optical  components  1 

11.  Tapes,  removable  drives,  plotter  pens,  1 

plotter  paper 


V 
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Support  Equipment .  Table  52  lists  the  ranking  and 
frequency  count  of  support  equipment  factors  unique  to 
automated  cartographic  or  photogrammetric  systems  as 
provided  by  the  DMA  experts  in  round  two. 


Table  52 

Support  Equipment  Factors  Unique 

to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  Two  Delphi  Survey 


Frequency 


1.  Calibration  of  support  and  test  equipment  16 

2.  Design  for  easily  replaced  logical  units,  15 

which  are  easily  repaired 

3.  Calibration  of  photogrammetric  equipment  is  11 

critical 

4.  Identification  of  support  and  test  equipment  10 

5.  Electronic  circuit  board  testing  and  9 

analysis  equipment 

6.  Optical  alignment  equipment  2 

7.  Screen  luminance  measuring  equipment  1 

8.  Infreometers  0 

9.  Resolution  targets  0 

10.  Signal  generators  0 

11.  Unique  IBM  gear  0 
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Technical  Data.  Table  53  lists  the  ranking  and 
frequency  count  of  technical,  data  factors  unique  to 
automated  cartographic  or  photogrammetric  systems  as 
provided  by  the  DMA  experts  in  round  two. 

Table  53 

Technical  Data  Factors  Unique 
to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  Two  Delphi  Survey 


Frequency 


1.  Maintenance  and  diagnostics  documentation  10 

2.  Software  documentation  and  interface  manuals  10 

3.  Communicate  with  contractor  when  developing  9 

manuals  used  for  production  and  maintenance 

4.  Critical/Preliminary  design  review  of  O&M  8 

and  Logistics  Plan  Equipment  operations 

manuals 

5.  Lists  of  delivered  hardware  and  software  items  5 

6.  COTS  equipment  has  impact  on  ability  to  5 

procure  adequate  technical  data  due  to 
proprietary  claims 

7.  Application  software  should  be  included  in  4 

Technical  data 

8.  Hardware  maintenance  manuals  4 

9.  Incorporate  operations  manuals  into  technical  4 

data  requirements 

10.  Mathematical  models  3 

11.  Schematics  3 

12.  Users  Guides  2 

13.  Stage  calibration  software  1 

14.  Configuration  items  index  0 

15.  Monitor  luminance  and  resolution  0 

16.  Quality  assurance  0 

17.  Technical  data  should  be  available  on  0 

microfiche 
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Training .  Table  54  lists  the  ranking  and  frequency 


count  of  training  factors  unique  to  automated  cartographic 
or  photogrammetric  systems  as  provided  by  the  DMA  experts  in 
round  two. 

Table  54 

Training  Factors  Unique 
to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  Two  Delphi  Survey 


Frequency 

1.  Critical/preliminary  design  review  of  8 

training  plans 

2.  In  house  hardware/software  training  must  be  8 

in  depth 

3 .  Need  preparatory  training  7 

4.  Need  segment  specific  training  7 

5.  Need  full -up  production  training  6 

6.  Require  more  than  two  maintenance  personnel  be  6 

trained  for  any  given  system 

7.  Self  paced  training  courses  (Auto  tutorials)  6 

8.  Need  prerequisite  training  to  determine  level  4 

of  readiness 

9.  Requirement  for  computer  skills  in  addition  to  4 

cartographic  skills 

10.  Training  should  be  developed  and  executed  by  4 

experienced  cartographic  and  photogrammetric 
personnel  because  training  often  is  developed 

from  scratch 

11.  Analytical  thinking  3 

12.  Implement  computer-assisted  training  3 

13.  Software  maintenance  requires  training  in  3 

photogrammetric  skills 

14.  Need  follow-on  training  2 

15.  Require  contractor  to  train  more  DMA  personnel  2 

initially  for  support  activities 

16.  Feature  identification  and  extraction  1 

17.  Fine  motor  skills  training  1 


Computer  Resources  Support .  Table  55  lists  the  ranking 


and  frequency  count  of  computer  resources  support  factors 
unique  to  automated  cartographic  or  photogrammetric  systems 
as  provided  by  the  DMA  experts  in  round  two. 


Table  55 

Computer  Resources  Support  Factors  Unique 

to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  Two  Delphi  Survey 


Frequency 


1.  Diagnostic  software  15 

2.  Flexibility,  expandability,  peak  capacity,  12 

and  fault  isolation  to  LRU 

3.  Need  a  development  test  facility  to  test  12 

software  code  fixes  and  upgrades  instead  of 

tying  up  production  equipment 

4.  Automated  support  and  test  equipment  9 

5.  Operating  system  upgrades  should  be  possible  9 

6.  COTS  packages  should  be  used  in  lieu  of  6 

special  purpose  applications  software 

7.  Image  processing  5 

8.  Hardware  and  software  should  be  considered  4 

as  separate  issues 

9.  Support  for  multiple  brands  of  computers  3 


r 
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Facilities .  Table  56  lists  the  ranking  and  frequency 


count  of  facilities  factors  unique  to  automated  cartographic 
or  photogrammetric  systems  as  provided  by  the  DMA  experts  in 
round  two. 

* 

Table  56 

Facilities  Factors  Unique 
to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  Two  Delphi  Survey 


Frequency 


1.  Designed  for  human  occupation  not  just  18 

machines 

2.  Clean  environment  12 

3.  Space  requirements  7 

4.  Security  emanations  6 

5.  Temperature  control  5 

6.  Electrical  requirements  5 

7.  Humidity  control  5 

8.  Fire  protection  requirements  4 

9.  Lighting  requirements  3 

10.  Facility  preparation  takes  too  long  creating  2 

disconnect  between  development  and  facility 
requirements 

11.  Personal  storage  areas  2 

12.  Vibration  control  2 

13.  Access  ramps  1 

14.  Noise  reduction  methods  1 

15.  Special  handling  requirements  1 

16.  Raised  floor  requirements  0 

17.  Static  electricity  reduction  techniques  0 

18.  Weight  requirements  0 
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Packaging ,  Handling .  Storage ,  &  Transportation.  Table 


57  lists  the  ranking  and  frequency  count  of  packaging, 
handling,  storage,  &  transportation  factors  unique  to 
automated  cartographic  or  photogrammetric  systems  as 
provided  by  the  DMA  experts  in  round  two. 


Table  57 

Packaging,  Handling,  Storage,  &  Transportation  Factors 

Unique  to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  Two  Delphi  Survey 


Freguency 


1.  Security  of  classified  items  13 

2.  Equipment  size  and  weight  10 

3.  Shipment  of  component  items  from  repair  depot  8 

to  customer 

4.  Storage  facilities  8 

5.  Packaging  materials  for  optical  equipment  7 

6.  Mode  of  transportation  6 

7.  Need  a  special  re-usable  container  for  6 

boards,  power  supplies,  etc 

8.  Shipment  of  initial  equipment  from  contractor  5 

to  customer 

9.  Personnel  resources  3 

10.  Air  ride  trailers  2 
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Design  Interface .  Table  58  lists  the  ranking  and 
frequency  count  of  design  interface  factors  unique  to 


automated  cartographic  or  photogrammetric  systems  as 
provided  by  the  DMA  experts  in  round  two. 

Table  58 

Design  Interface  Factors  Unique 
to 

Automated  Cartographic  or  Photogrammetric  Systems 
--  Round  Two  Delphi  Survey 


Frequency 

1.  Function  of  overall  system  design  19 

2.  Reliability/Maintainability  requirements  17 

3.  Need  to  incorporate  this  element  early  in  11 

the  life  cycle 

4.  Useful  system  life  10 

5.  Mean  time  between  failure  8 

6.  Mean  time  to  repair  5 

7.  Mean  time  to  restore  3 
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